-- Logs begin at Thu 2022-09-01 11:24:43 UTC, end at Thu 2022-09-01 12:20:28 UTC. -- Sep 01 11:56:11.615554 dv-cinder-839768-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 11:56:11.626258 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 11:56:11 2022] *** Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: nodename: dv-cinder-839768-4-iscsi Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: machine: x86_64 Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: clock source: unix Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: pcre jit disabled Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: detected number of CPU cores: 4 Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: current working directory: / Sep 01 11:56:11.628753 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: detected binary path: /usr/bin/uwsgi-core Sep 01 11:56:11.629337 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: your processes number limit is 47680 Sep 01 11:56:11.629337 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: your memory page size is 4096 bytes Sep 01 11:56:11.629337 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: detected max file descriptor number: 2048 Sep 01 11:56:11.629337 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: lock engine: pthread robust mutexes Sep 01 11:56:11.629337 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: thunder lock: enabled Sep 01 11:56:11.629337 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 11:56:11.629337 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 11:56:11.668270 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: Python main interpreter initialized at 0x55b9529a59c0 Sep 01 11:56:11.668270 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: python threads support enabled Sep 01 11:56:11.668270 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: your server socket listen backlog is limited to 100 connections Sep 01 11:56:11.668270 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: your mercy for graceful operations on workers is 90 seconds Sep 01 11:56:11.668663 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 11:56:11.668738 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: *** Operational MODE: preforking *** Sep 01 11:56:11.669005 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: *** uWSGI is running in multiple interpreter mode *** Sep 01 11:56:11.669005 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: spawned uWSGI master process (pid: 81849) Sep 01 11:56:11.669229 dv-cinder-839768-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 11:56:11.669846 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: spawned uWSGI worker 1 (pid: 81861, cores: 1) Sep 01 11:56:11.670326 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: spawned uWSGI worker 2 (pid: 81862, cores: 1) Sep 01 11:56:11.670722 dv-cinder-839768-4-iscsi devstack@keystone.service[81849]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 11:56:16.004127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 11:56:16.004127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: from cryptography.utils import int_from_bytes Sep 01 11:56:16.004913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 11:56:16.004913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: from cryptography.utils import int_from_bytes Sep 01 11:56:16.004913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 11:56:16.004913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: from cryptography.utils import int_from_bytes Sep 01 11:56:16.005414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 11:56:16.005414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: from cryptography.utils import int_from_bytes Sep 01 11:56:16.055319 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 11:56:16.055519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 11:56:16.055519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 11:56:16.055805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 11:56:16.055935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] command line args: [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 11:56:16.056045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] command line args: [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 11:56:16.056176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 11:56:16.056299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 11:56:16.056423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ================================================================================ {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 11:56:16.056553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ================================================================================ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 11:56:16.056699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] admin_token = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.056822 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] admin_token = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.056946 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] config_dir = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.057079 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config_dir = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.057210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.057341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.057523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] config_source = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.057716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] config_source = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.057884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] debug = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] debug = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.058944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059355 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] list_limit = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] log_config_append = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_config_append = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059851 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.059959 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060047 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] log_dir = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_dir = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060251 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] log_file = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060329 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060808 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.060952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061031 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061237 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061895 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.061970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.062967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.063054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.063128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.063252 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.063330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.063514 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.063596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] publish_errors = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064229 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] publish_errors = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064524 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064658 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064887 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.064965 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065037 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] standard_threads = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065116 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] standard_threads = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065194 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065572 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065734 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] use_journal = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_journal = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] use_json = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.065972 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_json = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.066050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] use_stderr = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.066211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_stderr = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.066299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] use_syslog = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.066372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] use_syslog = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.066443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.066519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 11:56:16.066629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.066720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.066868 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.066948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067207 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067843 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.067923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068567 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068811 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.068900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069289 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.069876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.070036 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.070115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.070545 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.070641 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.070784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.070863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071037 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071117 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071760 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.071884 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072503 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072747 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.072975 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.073934 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074269 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074635 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074715 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074877 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.074956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075354 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075602 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075721 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075840 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.075936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076070 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076565 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.076961 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077076 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077304 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.077956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078078 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078442 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.078931 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079047 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079404 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079852 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.079967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080302 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080415 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080775 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.080891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.081012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.081164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.081298 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.081433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.081552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.081715 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.081834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.082010 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.082202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.082384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.082547 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.082703 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.082826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.082980 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.083104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.083231 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.083349 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.083472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.083589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.083753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.083875 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084597 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.084957 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.085993 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086238 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086600 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.086947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.087064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.087191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.087317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.087457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.087598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.087735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.087883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088525 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088678 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088810 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.088965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.089107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.089260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.089420 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.089540 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.089694 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.089814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.089930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090047 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090427 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090785 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.090901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.091952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.092935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.093152 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.093339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.093490 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.093669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] auth.external = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.093790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.external = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.093914 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094584 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] auth.password = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.password = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.094993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] auth.token = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.095111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] auth.token = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.095237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.095357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.095474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.095627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.095753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.095905 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096188 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.096905 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097022 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] credential.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097256 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097498 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.097893 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098135 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098783 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.098933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.099170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.099299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.099456 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.099582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.099735 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.099862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.100027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.100181 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.100336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.100458 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.100612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.100745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.100901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.101023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.101177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.101177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.101451 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.101587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.101783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.101783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102206 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102365 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102365 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.102876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.103780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104231 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104231 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104914 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.104914 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.105135 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.105262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.105380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.105380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.105613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.105613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.105903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106430 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106547 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106664 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106664 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.106894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107246 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.107975 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108329 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108329 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108651 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108651 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.108878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109760 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.109997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.110113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.110241 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.110369 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.110973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111091 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.111927 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112284 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.112884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113008 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113127 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113742 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.113978 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114352 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114604 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.114953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115193 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.115910 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116263 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116379 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116495 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116621 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116856 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.116972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117329 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117449 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.117957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118793 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.118918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119353 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119831 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.119948 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120541 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.120890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121015 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121250 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.121976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.122995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.123186 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.123307 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.123424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.123540 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.123657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.123772 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.123897 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124356 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.124950 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125184 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125308 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125544 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.125935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.126065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.126908 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.126908 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127724 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127840 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.127964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128445 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128562 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.128930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.129156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.129274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.129400 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.129517 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.129652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] role.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.129781 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.129900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] role.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130142 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130260 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130608 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.130952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131067 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131302 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.131896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.132936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133055 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133508 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.133921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134751 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.134992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.135930 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136279 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136279 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.136974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137098 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137455 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.137957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138191 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138547 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138667 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.138898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139014 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139604 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.139947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140181 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140296 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140793 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.140913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141146 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141799 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.141917 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142407 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142407 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.142943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.connection = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143407 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.143874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144337 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144694 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.144930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145046 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145279 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145403 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81862) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.145522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 11:56:16.146509 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81861) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 11:56:16.171083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.171310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.171535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.172325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.172555 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.172780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.180719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.181674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.209929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.210155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.210592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.210825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.265281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.267637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.314929 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.315165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.315672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.315898 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.473889 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.474131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.474356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.474580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.474795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.475008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.475240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.475474 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.475700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.475936 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.476128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.476340 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.498486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.498666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.498750 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.498929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499018 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499094 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.499812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.511204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.511685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.524556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.524714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.524797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.524960 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525848 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81861) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.525930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.526036 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81862) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 11:56:16.542549 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.542754 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 11:56:16.542995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.543089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.543211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 11:56:16.543297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 11:56:16.543546 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.543628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.543745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 11:56:16.543829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.544061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.544274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.544357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.544544 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.544809 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.545075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.554427 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.554705 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.554904 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.555003 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.555184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.555295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.555458 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.555746 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.562223 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.562700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.565743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.566030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.566172 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.566317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.566457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.566599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.566751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.566883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.567010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.567145 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.567295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.567420 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.567558 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.567692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.567937 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.568029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.568214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.568298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.568478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.568560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.568753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.569023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.591346 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.591743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.592809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.593089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.593211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.593495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.597668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.597950 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.598196 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.598293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.598482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.598771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.604671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.604953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.605078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.605365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.608908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.609196 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.609435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.609536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.609759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.609845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.610035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.610118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.610318 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.611119 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.611425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.611698 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.611969 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.612588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.612883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.613168 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.629198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.629483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.629761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.630055 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.633318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.634039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.636393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.636687 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.637262 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.637563 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.640952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.641830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.644069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.645118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.647135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.647433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.647723 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.648009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.648246 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.648397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.648563 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.648683 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.648849 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.649135 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.649424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.649727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.664539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.664832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.665103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.665368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.665642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.665942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.666221 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.666478 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.672132 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.673881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.674338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.675415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.675881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.676161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.676952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.677231 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.678849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.679133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.679418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.679705 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.680001 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.680081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.680284 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.680567 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.680849 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.681148 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.691433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.691719 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.692002 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.692286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.692643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.692743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.692848 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.692955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.693135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.693241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.693526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.693846 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.694124 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.694399 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.709369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.710597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.712428 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.713514 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.715333 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.716422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.716769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.718035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.718293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.718561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.718853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.719511 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.719827 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.720839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.726957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.727413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.728432 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.729017 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.731337 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.731629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.731918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.732011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.732197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.732281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.732479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.732753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.740175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.740475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.740761 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.740850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.741050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.741328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.747697 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81861) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.747983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.748142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81862) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 11:56:16.748267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.748428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.748547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.748709 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.748822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.748983 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.749095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.749257 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.749379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.749531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.749680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.749833 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.749963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.750113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.750240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.750497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.750579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.750759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.750844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.751025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.751111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.751328 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81861) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.751428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.751597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.751873 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81862) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 11:56:16.786990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b9529a59c0 pid: 81862 (default app) Sep 01 11:56:16.787562 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b9529a59c0 pid: 81861 (default app) Sep 01 11:56:16.790393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd98169-cb0e-49ad-a318-7f9afbbc5c87 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:16.790682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd98169-cb0e-49ad-a318-7f9afbbc5c87 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:16.790955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd98169-cb0e-49ad-a318-7f9afbbc5c87 None None] PATH_INFO: `/v3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:16.795276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Thu Sep 1 11:56:16 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:24.603809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee84d-d077-43ab-9bd2-69cd88b1f0a5 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:24.604400 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee84d-d077-43ab-9bd2-69cd88b1f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:24.604400 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee84d-d077-43ab-9bd2-69cd88b1f0a5 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:24.608717 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:24.614955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:24.615458 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:24.615556 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:24.685241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: INFO dbcounter [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] Registered counter for database keystone Sep 01 11:56:24.706858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG oslo_db.sqlalchemy.engines [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81862) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 11:56:24.713271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writer thread running {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 11:56:24.898699 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG passlib.handlers.bcrypt [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81862) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 11:56:24.899099 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG passlib.handlers.bcrypt [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81862) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 11:56:24.960815 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:24.962363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:25.095850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 11:56:25.395058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dee84d-d077-43ab-9bd2-69cd88b1f0a5 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:25.465486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: INFO dbcounter [None req-33dee84d-d077-43ab-9bd2-69cd88b1f0a5 None None] Registered counter for database keystone Sep 01 11:56:25.484934 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG oslo_db.sqlalchemy.engines [None req-33dee84d-d077-43ab-9bd2-69cd88b1f0a5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81861) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 11:56:25.491693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writer thread running {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 11:56:25.612792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dee84d-d077-43ab-9bd2-69cd88b1f0a5 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:25.617511 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe44aaa-2278-4708-8703-4c7fd2f4340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:25.618200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44aaa-2278-4708-8703-4c7fd2f4340a admin admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:25.618493 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44aaa-2278-4708-8703-4c7fd2f4340a admin admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:25.618759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44aaa-2278-4708-8703-4c7fd2f4340a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:25.628249 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe44aaa-2278-4708-8703-4c7fd2f4340a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:25.680703 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bfe44aaa-2278-4708-8703-4c7fd2f4340a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 11:56:25.680971 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe44aaa-2278-4708-8703-4c7fd2f4340a admin admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:25.691603 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Sep 1 11:56:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 11:56:25.697939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:25.713500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3b1da-e79b-4047-9c2e-2913dd743545 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:25.718026 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248db529-7945-4540-8990-7f2269b4aee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:25.719176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db529-7945-4540-8990-7f2269b4aee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:25.719657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db529-7945-4540-8990-7f2269b4aee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:25.720541 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db529-7945-4540-8990-7f2269b4aee5 admin admin] PATH_INFO: `/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:25.722269 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248db529-7945-4540-8990-7f2269b4aee5 admin admin] RBAC: Authorizing `identity:get_project(project_id=d6c17f90daea4fc3b1b162a692b4ec70)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:25.774868 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: WARNING keystone.common.rbac_enforcer.enforcer [None req-248db529-7945-4540-8990-7f2269b4aee5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 11:56:25.775215 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248db529-7945-4540-8990-7f2269b4aee5 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:25.779925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Sep 1 11:56:25 2022] GET /identity/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:27.655046 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f471-dd9b-4314-952b-a3a6b660c716 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:27.655576 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f471-dd9b-4314-952b-a3a6b660c716 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:27.655684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47f471-dd9b-4314-952b-a3a6b660c716 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:27.659751 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:56:27.665191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd46e1-ae36-46e0-9c41-1132dba5c46a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:27.665665 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd46e1-ae36-46e0-9c41-1132dba5c46a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:27.665765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd46e1-ae36-46e0-9c41-1132dba5c46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:27.685488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-45fd46e1-ae36-46e0-9c41-1132dba5c46a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:27.687034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-45fd46e1-ae36-46e0-9c41-1132dba5c46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:27.695603 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 11:56:27.964862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47f471-dd9b-4314-952b-a3a6b660c716 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:27.978327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47f471-dd9b-4314-952b-a3a6b660c716 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:27.982629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79bb6f-466b-4c47-ae71-e83ff42c3a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:27.983251 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79bb6f-466b-4c47-ae71-e83ff42c3a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:27.983534 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79bb6f-466b-4c47-ae71-e83ff42c3a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:27.983813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79bb6f-466b-4c47-ae71-e83ff42c3a00 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:27.992447 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79bb6f-466b-4c47-ae71-e83ff42c3a00 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:27.994268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79bb6f-466b-4c47-ae71-e83ff42c3a00 admin admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:28.004663 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Sep 1 11:56:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 11:56:28.010449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd46e1-ae36-46e0-9c41-1132dba5c46a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:28.019981 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd46e1-ae36-46e0-9c41-1132dba5c46a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:28.024302 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c5383-54a9-4960-a822-d2b288fe5d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:28.024934 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5383-54a9-4960-a822-d2b288fe5d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:28.025212 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5383-54a9-4960-a822-d2b288fe5d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:28.025484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5383-54a9-4960-a822-d2b288fe5d58 admin admin] PATH_INFO: `/v3/users/11b1bb7d9ce240aab79c21f829ee58ed` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:28.026837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c5383-54a9-4960-a822-d2b288fe5d58 admin admin] RBAC: Authorizing `identity:get_user(user_id=11b1bb7d9ce240aab79c21f829ee58ed)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:28.028666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c5383-54a9-4960-a822-d2b288fe5d58 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:28.030236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Thu Sep 1 11:56:28 2022] GET /identity/v3/users/11b1bb7d9ce240aab79c21f829ee58ed => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:30.040235 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78654775-b79e-4ada-9d6e-0b80da61b5c7 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.040782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78654775-b79e-4ada-9d6e-0b80da61b5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.040782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78654775-b79e-4ada-9d6e-0b80da61b5c7 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.042396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:30.047481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f073-60cd-4b56-bb41-f6bcff130a25 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.048533 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f073-60cd-4b56-bb41-f6bcff130a25 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.048623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f073-60cd-4b56-bb41-f6bcff130a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.066888 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0ce7f073-60cd-4b56-bb41-f6bcff130a25 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:30.068071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0ce7f073-60cd-4b56-bb41-f6bcff130a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:30.092782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2ef24-0970-4331-8e56-55bdce1f1cf4 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.093157 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2ef24-0970-4331-8e56-55bdce1f1cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.093500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2ef24-0970-4331-8e56-55bdce1f1cf4 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.095628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:56:30.100858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.101140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.101409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.142985 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG passlib.handlers.bcrypt [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81861) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 11:56:30.143367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG passlib.handlers.bcrypt [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81861) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 11:56:30.151042 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:30.192297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:30.193939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:30.296861 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:30.419387 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7f073-60cd-4b56-bb41-f6bcff130a25 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:30.451397 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:30.548786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7f073-60cd-4b56-bb41-f6bcff130a25 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:30.569497 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:30.584409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:30.585064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.585347 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.585641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.598812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:30.607711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:30.610067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:30.612409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:56:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:30.615962 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:30.682080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebea20b-cf8b-4e44-a7ff-cf5ccd62136e None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:30.684622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:30.725654 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:30.735363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:30.736532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.737013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.737348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.745288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163af248-13e1-4d4b-bf6f-2c706b2637c1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:30.785397 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:30.786048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.786324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.786603 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.787282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:30.787577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:30.788982 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:30.802414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:30.807784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:30.816920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:30.818538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:30.923517 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d61f3-dd32-4c87-aec3-af5be1f7f34a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:30.934622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 8/16] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:56:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:56:30.940416 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:30.945874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:30.960593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:30.961243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:30.961529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:30.961841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] PATH_INFO: `/v3/users/11b1bb7d9ce240aab79c21f829ee58ed` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:30.963026 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] RBAC: Authorizing `identity:get_user(user_id=11b1bb7d9ce240aab79c21f829ee58ed)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:30.964529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:30.966208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:30 2022] GET /identity/v3/users/11b1bb7d9ce240aab79c21f829ee58ed => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:30.973314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:30.989012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:31.056707 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b3435-85bd-4765-a99b-17be2ccd1d46 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:31.087475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb90be5-4de9-49c1-b48d-02ac495e45c6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:31.092998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:31.093803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:31.094089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:31.094367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:31.116230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:31.117842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:31.125197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:31.126479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:31.126774 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:31.127048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:31.128358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:31.129914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:31.131600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 10/18] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:31.138320 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:56:31.139581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:31.141307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:56:31.142941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:56:31.142941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:56:31.142941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:56:31.142941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 9/19] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:56:31.147007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:31.214045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:31.237401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:31.259688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d19ba-6471-468b-922b-e4d9c81e1313 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:31.267239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26149d-5a88-43a4-af88-5848571684dc None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:31.300605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3efb6e-7d41-40be-b947-2e3cd9a4be33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:31.301240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efb6e-7d41-40be-b947-2e3cd9a4be33 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:31.301519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efb6e-7d41-40be-b947-2e3cd9a4be33 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:31.301829 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3efb6e-7d41-40be-b947-2e3cd9a4be33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:31.302574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3efb6e-7d41-40be-b947-2e3cd9a4be33 None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.domain.id=default, user.id=11b1bb7d9ce240aab79c21f829ee58ed) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:31.302886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3efb6e-7d41-40be-b947-2e3cd9a4be33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:31.304386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3efb6e-7d41-40be-b947-2e3cd9a4be33 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:31.305809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ec67e-b23f-4a5c-ac98-ec7c1bdb2719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:31.306439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec67e-b23f-4a5c-ac98-ec7c1bdb2719 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:31.306772 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec67e-b23f-4a5c-ac98-ec7c1bdb2719 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:31.306993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec67e-b23f-4a5c-ac98-ec7c1bdb2719 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:31.307652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec67e-b23f-4a5c-ac98-ec7c1bdb2719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:31.307957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec67e-b23f-4a5c-ac98-ec7c1bdb2719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:31.309365 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec67e-b23f-4a5c-ac98-ec7c1bdb2719 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:31.311965 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 11/20] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Sep 1 11:56:31 2022] GET /identity/v3/role_assignments?user.id=11b1bb7d9ce240aab79c21f829ee58ed&scope.domain.id=default&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 233 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:31.334061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 10/21] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:33.340166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4190-0b7f-44eb-ae68-a1fbbe744466 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:33.340964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4190-0b7f-44eb-ae68-a1fbbe744466 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:33.340964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4190-0b7f-44eb-ae68-a1fbbe744466 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:33.342090 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 12/22] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:56:33.347913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b609-edae-48a0-98a1-93e4e2580139 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:33.348593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b609-edae-48a0-98a1-93e4e2580139 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:33.348593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b609-edae-48a0-98a1-93e4e2580139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:33.374668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-7636b609-edae-48a0-98a1-93e4e2580139 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:33.376443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7636b609-edae-48a0-98a1-93e4e2580139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:33.473233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 11/23] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:33.998933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a4190-0b7f-44eb-ae68-a1fbbe744466 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:34.134259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a4190-0b7f-44eb-ae68-a1fbbe744466 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:34.186068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:34.186718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:34.187002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:34.187300 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:34.209115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:34.214750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:34.216275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:34.217676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 13/24] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:56:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:56:34.222364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636b609-edae-48a0-98a1-93e4e2580139 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:34.285992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:34.354846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636b609-edae-48a0-98a1-93e4e2580139 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:34.406614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:34.407800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:34.408080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:34.408353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:34.409044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:34.409351 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:34.410807 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:34.430370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 12/25] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:34.436313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:34.440778 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:34.605481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49e2cc-c565-494c-95bb-4580eb0d4853 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:34.647135 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b81e4-de66-41be-85be-89299655e75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:34.647824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b81e4-de66-41be-85be-89299655e75f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:34.648108 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b81e4-de66-41be-85be-89299655e75f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:34.653645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b81e4-de66-41be-85be-89299655e75f None admin] PATH_INFO: `/v3/users/11b1bb7d9ce240aab79c21f829ee58ed` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:34.653645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b81e4-de66-41be-85be-89299655e75f None admin] RBAC: Authorizing `identity:get_user(user_id=11b1bb7d9ce240aab79c21f829ee58ed)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:34.654902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b81e4-de66-41be-85be-89299655e75f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:34.657009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 14/26] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:34 2022] GET /identity/v3/users/11b1bb7d9ce240aab79c21f829ee58ed => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:34.662967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:34.794013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41dee2-8869-40b5-a7a3-1fd15e04671d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:34.808169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:34.837921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ad62b-0cbc-4906-889d-eaa155c7f3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:34.838609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ad62b-0cbc-4906-889d-eaa155c7f3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:34.838918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ad62b-0cbc-4906-889d-eaa155c7f3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:34.839194 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ad62b-0cbc-4906-889d-eaa155c7f3c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:34.842211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ad62b-0cbc-4906-889d-eaa155c7f3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:34.843796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ad62b-0cbc-4906-889d-eaa155c7f3c8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:34.847113 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 13/27] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:34.852745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b81e4-de66-41be-85be-89299655e75f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:34.949921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:34.990035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b81e4-de66-41be-85be-89299655e75f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:35.042274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87d8e9-cd24-42d4-baf9-69c42e5fdc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:35.043012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d8e9-cd24-42d4-baf9-69c42e5fdc0f None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:35.043304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d8e9-cd24-42d4-baf9-69c42e5fdc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:35.043585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d8e9-cd24-42d4-baf9-69c42e5fdc0f None admin] PATH_INFO: `/v3/domains/default/users/11b1bb7d9ce240aab79c21f829ee58ed/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:35.045675 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87d8e9-cd24-42d4-baf9-69c42e5fdc0f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=11b1bb7d9ce240aab79c21f829ee58ed, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:35.047171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87d8e9-cd24-42d4-baf9-69c42e5fdc0f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:35.281030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 15/28] 10.222.0.48 () {58 vars in 1361 bytes} [Thu Sep 1 11:56:34 2022] PUT /identity/v3/domains/default/users/11b1bb7d9ce240aab79c21f829ee58ed/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:56:35.372933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f4600-5c1a-4a27-bc13-a58ebd0e81c5 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:35.374312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f4600-5c1a-4a27-bc13-a58ebd0e81c5 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:35.375023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f4600-5c1a-4a27-bc13-a58ebd0e81c5 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:35.380463 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 14/29] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:56:35.389641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3015e-dea7-46fb-8ea4-dd036634b562 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:35.390110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3015e-dea7-46fb-8ea4-dd036634b562 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:35.390499 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3015e-dea7-46fb-8ea4-dd036634b562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:35.419027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5bc3015e-dea7-46fb-8ea4-dd036634b562 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:35.420738 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5bc3015e-dea7-46fb-8ea4-dd036634b562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:35.497462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ad63-05bc-4ff7-b702-f0163d26c193 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:35.498411 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ad63-05bc-4ff7-b702-f0163d26c193 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:35.498988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ad63-05bc-4ff7-b702-f0163d26c193 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:35.501569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 15/30] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:35.507276 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9182-f77e-4714-bf0b-01aa05e1bcde None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:35.508075 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9182-f77e-4714-bf0b-01aa05e1bcde None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:35.508550 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a9182-f77e-4714-bf0b-01aa05e1bcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:35.545078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 16/31] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:35.546526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-157a9182-f77e-4714-bf0b-01aa05e1bcde None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:35.548338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-157a9182-f77e-4714-bf0b-01aa05e1bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:35.635438 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 16/32] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:35.846828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3015e-dea7-46fb-8ea4-dd036634b562 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:35.937776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:35.991819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3015e-dea7-46fb-8ea4-dd036634b562 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:36.050413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:36.051093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.051381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.051482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a9182-f77e-4714-bf0b-01aa05e1bcde None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:36.051661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.064659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:36.066499 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:56:36.089235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:36.090182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:56:36.090984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:56:36.090984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:56:36.090984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:36.096906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 17/33] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:36.102884 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:36.226285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ddb05-82c4-4c25-bb46-c4af72d940ed None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:36.253977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:36.317324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9aee0-3d8c-462a-8c64-3314ab4c8555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:36.318410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9aee0-3d8c-462a-8c64-3314ab4c8555 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.320865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9aee0-3d8c-462a-8c64-3314ab4c8555 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.321450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9aee0-3d8c-462a-8c64-3314ab4c8555 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.322426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9aee0-3d8c-462a-8c64-3314ab4c8555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:36.327043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9aee0-3d8c-462a-8c64-3314ab4c8555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:36.329505 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9aee0-3d8c-462a-8c64-3314ab4c8555 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:36.350741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a9182-f77e-4714-bf0b-01aa05e1bcde None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:36.378220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 18/34] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:36.379921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26459c45-fb6a-487a-b19b-972817c38418 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.380406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26459c45-fb6a-487a-b19b-972817c38418 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.380949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26459c45-fb6a-487a-b19b-972817c38418 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.382008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:36.382587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 19/35] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:36.387701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26459c45-fb6a-487a-b19b-972817c38418 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:36.417171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327758ac-65cd-4785-823c-2d4e7c4c0602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:36.418821 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327758ac-65cd-4785-823c-2d4e7c4c0602 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.419504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327758ac-65cd-4785-823c-2d4e7c4c0602 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.419983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327758ac-65cd-4785-823c-2d4e7c4c0602 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.421117 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327758ac-65cd-4785-823c-2d4e7c4c0602 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:36.435606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327758ac-65cd-4785-823c-2d4e7c4c0602 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:36.522394 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26459c45-fb6a-487a-b19b-972817c38418 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:36.565791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3bb05-7182-446a-a2d2-9b4e78615f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:36.567899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3bb05-7182-446a-a2d2-9b4e78615f8b None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.567899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3bb05-7182-446a-a2d2-9b4e78615f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.567899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3bb05-7182-446a-a2d2-9b4e78615f8b None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.567899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3bb05-7182-446a-a2d2-9b4e78615f8b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:36.569479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3bb05-7182-446a-a2d2-9b4e78615f8b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:36.676630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 17/37] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Sep 1 11:56:36 2022] POST /identity/v3/roles => generated 219 bytes in 626 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:56:36.681620 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 20/37] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:56:36 2022] POST /identity/v3/projects => generated 301 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:56:36.683574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43feb4c-d4d3-45a7-b059-97bd22180d14 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.683915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43feb4c-d4d3-45a7-b059-97bd22180d14 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.684196 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43feb4c-d4d3-45a7-b059-97bd22180d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.684471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8da85-41bd-461a-9043-b266f0dbd3d2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.685129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8da85-41bd-461a-9043-b266f0dbd3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.685854 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8da85-41bd-461a-9043-b266f0dbd3d2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.687818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 21/38] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:36.716205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b363616-5f40-41ca-bce8-b57e0e864b53 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.716863 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b363616-5f40-41ca-bce8-b57e0e864b53 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.717473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b363616-5f40-41ca-bce8-b57e0e864b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.725183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-a43feb4c-d4d3-45a7-b059-97bd22180d14 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:36.726987 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-a43feb4c-d4d3-45a7-b059-97bd22180d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:36.765126 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5b363616-5f40-41ca-bce8-b57e0e864b53 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:36.770262 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5b363616-5f40-41ca-bce8-b57e0e864b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:36.826061 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:36.893681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 18/39] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:36.926194 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431eded-be6f-4ef4-a20f-1919e2c0b5ea None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.926540 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431eded-be6f-4ef4-a20f-1919e2c0b5ea None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.926822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431eded-be6f-4ef4-a20f-1919e2c0b5ea None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.928371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 19/40] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:56:36.939497 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541419b-5edf-4783-b181-843e62e3aaa6 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:36.939807 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541419b-5edf-4783-b181-843e62e3aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:36.940092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541419b-5edf-4783-b181-843e62e3aaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:36.950493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 22/41] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:36.963252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-1541419b-5edf-4783-b181-843e62e3aaa6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:36.964580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-1541419b-5edf-4783-b181-843e62e3aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:37.046678 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 20/42] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:37.288819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b363616-5f40-41ca-bce8-b57e0e864b53 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:37.355405 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541419b-5edf-4783-b181-843e62e3aaa6 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:37.390785 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:37.485330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541419b-5edf-4783-b181-843e62e3aaa6 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:37.529483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:37.530527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:37.530816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:37.531334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:37.532621 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:37.534079 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:37.558306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b363616-5f40-41ca-bce8-b57e0e864b53 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:37.593850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:37.595121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:37.595309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:37.595589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:37.596294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:37.597752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:37.617798 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 21/43] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Sep 1 11:56:37 2022] POST /identity/v3/roles => generated 225 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:56:37.618646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:37.685667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 23/44] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Sep 1 11:56:37 2022] POST /identity/v3/roles => generated 223 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:56:37.752805 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:37.772569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f75f8-a5c8-492f-8a13-5a38e38fa0c0 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:37.818174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d60559-2178-4c50-8c8c-09c6ac27836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:37.819364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d60559-2178-4c50-8c8c-09c6ac27836c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:37.819890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d60559-2178-4c50-8c8c-09c6ac27836c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:37.820745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d60559-2178-4c50-8c8c-09c6ac27836c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:37.822468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d60559-2178-4c50-8c8c-09c6ac27836c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:37.824422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d60559-2178-4c50-8c8c-09c6ac27836c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:37.826581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 22/45] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:37.833261 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:37.856023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:37.950223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a1b1e-eeef-4c4e-9206-b4d215d2e8cd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:37.984824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabc2e8-ad7d-4454-824b-ffd9a93d6051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:37.985516 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc2e8-ad7d-4454-824b-ffd9a93d6051 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:37.985833 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc2e8-ad7d-4454-824b-ffd9a93d6051 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:37.986118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc2e8-ad7d-4454-824b-ffd9a93d6051 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:37.986781 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabc2e8-ad7d-4454-824b-ffd9a93d6051 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:37.988253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabc2e8-ad7d-4454-824b-ffd9a93d6051 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:38.089703 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 24/46] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:56:37 2022] POST /identity/v3/projects => generated 312 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:56:38.213007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384d10-29b0-4d01-a0ec-687d8b97e887 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:38.213306 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384d10-29b0-4d01-a0ec-687d8b97e887 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:38.213592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384d10-29b0-4d01-a0ec-687d8b97e887 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:38.215026 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 23/47] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:38.220873 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba525648-f390-47f3-9e3b-587581d706fa None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:38.221161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba525648-f390-47f3-9e3b-587581d706fa None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:38.221441 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba525648-f390-47f3-9e3b-587581d706fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:38.240375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ba525648-f390-47f3-9e3b-587581d706fa None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:38.241590 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ba525648-f390-47f3-9e3b-587581d706fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:38.305233 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:38.310570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 25/48] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:38.579866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384d10-29b0-4d01-a0ec-687d8b97e887 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:38.682665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384d10-29b0-4d01-a0ec-687d8b97e887 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:38.718896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:38.718896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:38.718896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:38.718896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:38.731396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:38.738440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:38.739458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:56:38.740523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 24/49] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:56:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:38.745666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba525648-f390-47f3-9e3b-587581d706fa None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:38.849393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba525648-f390-47f3-9e3b-587581d706fa None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:38.884024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:38.884641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:38.884931 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:38.885211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:38.885925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:38.886236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:38.887649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:38.896650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 26/50] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:38.902337 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:38.919498 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:39.011759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9cf33-040b-4ddf-97d1-28184dfcbc89 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:39.049481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:39.050153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:39.050436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:39.050712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] PATH_INFO: `/v3/users/11b1bb7d9ce240aab79c21f829ee58ed` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:39.052112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] RBAC: Authorizing `identity:get_user(user_id=11b1bb7d9ce240aab79c21f829ee58ed)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:39.053566 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:39.055212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 25/51] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:38 2022] GET /identity/v3/users/11b1bb7d9ce240aab79c21f829ee58ed => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:39.062050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:39.130575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:39.173620 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67406084-ea09-4af7-bb35-2bd13750bdbb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:39.211649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573e4b4-7438-4782-8793-acdcef670436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:39.212298 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573e4b4-7438-4782-8793-acdcef670436 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:39.212586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573e4b4-7438-4782-8793-acdcef670436 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:39.212865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573e4b4-7438-4782-8793-acdcef670436 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:39.214013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573e4b4-7438-4782-8793-acdcef670436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:39.215503 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573e4b4-7438-4782-8793-acdcef670436 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:39.217496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 27/52] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:39.223240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:39.336025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:39.354181 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad9ae0-f364-4b19-8455-c3779f7073c2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:39.391794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be974ac-6b98-4288-a3ee-e66f25211f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:39.392419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be974ac-6b98-4288-a3ee-e66f25211f18 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:39.392701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be974ac-6b98-4288-a3ee-e66f25211f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:39.392982 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be974ac-6b98-4288-a3ee-e66f25211f18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:39.394048 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be974ac-6b98-4288-a3ee-e66f25211f18 None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.domain.id=default, user.id=11b1bb7d9ce240aab79c21f829ee58ed) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:39.394048 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be974ac-6b98-4288-a3ee-e66f25211f18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:39.395552 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be974ac-6b98-4288-a3ee-e66f25211f18 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:39.404683 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 26/53] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Sep 1 11:56:39 2022] GET /identity/v3/role_assignments?user.id=11b1bb7d9ce240aab79c21f829ee58ed&scope.domain.id=default&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 530 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:40.023139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3bdf-5170-468c-9639-bc244d78592d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:40.023682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3bdf-5170-468c-9639-bc244d78592d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:40.023682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3bdf-5170-468c-9639-bc244d78592d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:40.025050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 28/54] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:40.031023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c52e-4647-4c5b-bae6-2ec959222a4e None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:40.031563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c52e-4647-4c5b-bae6-2ec959222a4e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:40.031647 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c52e-4647-4c5b-bae6-2ec959222a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:40.158777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-af76c52e-4647-4c5b-bae6-2ec959222a4e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:40.160007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-af76c52e-4647-4c5b-bae6-2ec959222a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:40.227390 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 27/55] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:40.494656 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df3bdf-5170-468c-9639-bc244d78592d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:40.594282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df3bdf-5170-468c-9639-bc244d78592d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:40.620848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:40.628855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fffcb-634c-4c9b-8840-237fb1259e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:40.635335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fffcb-634c-4c9b-8840-237fb1259e1c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:40.635335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fffcb-634c-4c9b-8840-237fb1259e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:40.635335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fffcb-634c-4c9b-8840-237fb1259e1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:40.635335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fffcb-634c-4c9b-8840-237fb1259e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:40.635335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fffcb-634c-4c9b-8840-237fb1259e1c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:40.635335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 29/56] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:40.639790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76c52e-4647-4c5b-bae6-2ec959222a4e None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:40.714345 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:40.736343 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76c52e-4647-4c5b-bae6-2ec959222a4e None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:40.769304 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb6cb8-4b3f-42e3-ace0-25640643a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:40.769949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb6cb8-4b3f-42e3-ace0-25640643a23a None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:40.770230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb6cb8-4b3f-42e3-ace0-25640643a23a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:40.770506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb6cb8-4b3f-42e3-ace0-25640643a23a None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:40.771150 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb6cb8-4b3f-42e3-ace0-25640643a23a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:40.772606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb6cb8-4b3f-42e3-ace0-25640643a23a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:40.868676 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 28/57] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Sep 1 11:56:40 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:56:42.776333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478180a9-d6b0-4289-a91e-e31b303945fb None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:42.776850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478180a9-d6b0-4289-a91e-e31b303945fb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:42.777024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478180a9-d6b0-4289-a91e-e31b303945fb None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:42.778453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 30/58] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:42.784277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce46e0-1185-43ab-826a-92fc38ef9932 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:42.784723 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce46e0-1185-43ab-826a-92fc38ef9932 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:42.784852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce46e0-1185-43ab-826a-92fc38ef9932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:42.803606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-c3ce46e0-1185-43ab-826a-92fc38ef9932 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:42.804963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-c3ce46e0-1185-43ab-826a-92fc38ef9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:42.870879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 29/59] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:43.141630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478180a9-d6b0-4289-a91e-e31b303945fb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:43.246177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478180a9-d6b0-4289-a91e-e31b303945fb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:43.279658 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0db58a-8384-4571-a29d-471a0e68c708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:43.280296 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0db58a-8384-4571-a29d-471a0e68c708 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:43.280585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0db58a-8384-4571-a29d-471a0e68c708 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:43.280862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0db58a-8384-4571-a29d-471a0e68c708 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:43.282568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0db58a-8384-4571-a29d-471a0e68c708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:43.284029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0db58a-8384-4571-a29d-471a0e68c708 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:43.285623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 31/60] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:43.293904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce46e0-1185-43ab-826a-92fc38ef9932 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:43.355291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:43.397462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce46e0-1185-43ab-826a-92fc38ef9932 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:43.431291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d575c6-f478-499a-892b-35b55e2479a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:43.431903 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d575c6-f478-499a-892b-35b55e2479a6 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:43.432187 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d575c6-f478-499a-892b-35b55e2479a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:43.432460 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d575c6-f478-499a-892b-35b55e2479a6 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:43.433111 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d575c6-f478-499a-892b-35b55e2479a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:43.434590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d575c6-f478-499a-892b-35b55e2479a6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:43.528019 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 30/61] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:56:43 2022] POST /identity/v3/users => generated 275 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:56:47.021347 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebaff8-c2fd-4d3d-a30c-4c0a09820d9b None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.022873 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebaff8-c2fd-4d3d-a30c-4c0a09820d9b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.023641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebaff8-c2fd-4d3d-a30c-4c0a09820d9b None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.026845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 32/62] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:47.035846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd71718-dc63-4aac-abda-d735a92e32cf None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.037271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd71718-dc63-4aac-abda-d735a92e32cf None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.038165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd71718-dc63-4aac-abda-d735a92e32cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.075541 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-bdd71718-dc63-4aac-abda-d735a92e32cf None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:47.078070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-bdd71718-dc63-4aac-abda-d735a92e32cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:47.243398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 31/63] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:47.354403 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11261d-3a04-456e-8285-1e1f4ed060a5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.355409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11261d-3a04-456e-8285-1e1f4ed060a5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.356200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11261d-3a04-456e-8285-1e1f4ed060a5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.359048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 33/64] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:47.366001 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35cbb-ddb4-4c38-892a-5f0d7e037f8d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.366916 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35cbb-ddb4-4c38-892a-5f0d7e037f8d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.367666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35cbb-ddb4-4c38-892a-5f0d7e037f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.413774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-94d35cbb-ddb4-4c38-892a-5f0d7e037f8d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:47.416310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-94d35cbb-ddb4-4c38-892a-5f0d7e037f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:47.528021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:47.593411 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 32/65] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:47.709929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623408c-8500-4cc2-85a4-693d5c18bd1c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.710342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623408c-8500-4cc2-85a4-693d5c18bd1c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.710861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623408c-8500-4cc2-85a4-693d5c18bd1c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.713199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 34/66] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:47.725260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f29932-a7fb-46f2-8d98-5f21634b1813 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.725877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f29932-a7fb-46f2-8d98-5f21634b1813 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.726373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f29932-a7fb-46f2-8d98-5f21634b1813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.763159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-81f29932-a7fb-46f2-8d98-5f21634b1813 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:47.765071 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-81f29932-a7fb-46f2-8d98-5f21634b1813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:47.887380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446222-baaa-41a1-bc16-f908c162b336 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.887919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446222-baaa-41a1-bc16-f908c162b336 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.888402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db446222-baaa-41a1-bc16-f908c162b336 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.890868 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 35/67] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:47.893986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 33/68] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:47.903098 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cad8c-7fb3-4201-a0ca-6df822f950ed None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:47.903566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cad8c-7fb3-4201-a0ca-6df822f950ed None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:47.904054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cad8c-7fb3-4201-a0ca-6df822f950ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:47.915106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f29932-a7fb-46f2-8d98-5f21634b1813 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:47.945203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-737cad8c-7fb3-4201-a0ca-6df822f950ed None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:47.947182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-737cad8c-7fb3-4201-a0ca-6df822f950ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:48.030801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:48.040673 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 36/69] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:48.048321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1e4a0-b914-4f8f-8306-eacd835783a2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.048321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1e4a0-b914-4f8f-8306-eacd835783a2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.048321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1e4a0-b914-4f8f-8306-eacd835783a2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.048321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 37/70] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:48.048321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1e4a0-b914-4f8f-8306-eacd835783a2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.095934 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f29932-a7fb-46f2-8d98-5f21634b1813 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.101337 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:48.160151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1e4a0-b914-4f8f-8306-eacd835783a2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.163996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e59680-0233-4182-8d44-797571406b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:48.165396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e59680-0233-4182-8d44-797571406b36 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.166197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e59680-0233-4182-8d44-797571406b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.166955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e59680-0233-4182-8d44-797571406b36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.187374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e59680-0233-4182-8d44-797571406b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:48.196934 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:48.197569 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-74e59680-0233-4182-8d44-797571406b36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:48.197828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:48.198626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:48.199404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:56:48.200000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.200000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.200125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 34/71] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:48.201943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2d892-6258-4da1-8ae2-4112ad80aa99 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.202476 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2d892-6258-4da1-8ae2-4112ad80aa99 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.202979 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2d892-6258-4da1-8ae2-4112ad80aa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.216318 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:48.222669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:48.230248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 38/72] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:48.231301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.233029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-1ad2d892-6258-4da1-8ae2-4112ad80aa99 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:48.234447 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-1ad2d892-6258-4da1-8ae2-4112ad80aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:48.300277 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:48.345905 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 35/73] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:48.347056 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2d892-6258-4da1-8ae2-4112ad80aa99 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.361215 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4ca7-2091-488b-bbb0-ce118aff2fdf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.412278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:48.412609 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.413803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.413803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.431185 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:48.436794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:48.437732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:48.438987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:48.444009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 39/74] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:48.445842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.515148 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:48.527770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2d892-6258-4da1-8ae2-4112ad80aa99 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.587776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2c6aa-1387-448f-9713-04a70f1d0927 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.596263 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:48.597461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.598100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.598523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.599842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:48.600429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:48.603008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:48.605015 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:48.619599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 36/75] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:48.628255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.629200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:48.629941 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.630316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.630667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.631425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:48.631803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:48.633644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:48.643500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 40/76] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:48.644476 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.767477 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047e23b-32bc-458d-8b99-f0a3de9fe89c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.781590 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01af8cf-3dd1-4ff1-b9c6-ce58b3d68246 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.818490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:48.819622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.820374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.820874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.830130 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:48.830844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:48.831134 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:48.831416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:48.832094 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:48.832406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:48.833841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:48.837977 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:48.843792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:48.846204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:56:48.847686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 37/77] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:48.849918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.857752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 41/78] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:48.859213 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:48.877275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:48.910005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:48.973626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c62556-d822-4889-9623-7f5a3e161290 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:48.974419 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd0946-890d-4560-b7f3-5d4b1b6bb58b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.010725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.011398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.011707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.011988 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.026993 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.028484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.030258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 42/79] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:49.031626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.033059 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.033339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.033639 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.034551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.034986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.036247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.041659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 38/80] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:49.041659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.148127 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:49.150525 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:49.157886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb0530-b6f9-411c-85b5-ac51d9ca7bdb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.167402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c3e2-c516-44db-b3fe-1e7180f33550 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.207211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.207874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.208171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.208443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] PATH_INFO: `/v3/users/11b1bb7d9ce240aab79c21f829ee58ed` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.209744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] RBAC: Authorizing `identity:get_user(user_id=11b1bb7d9ce240aab79c21f829ee58ed)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.210615 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.211266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.211450 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.211832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.212367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.213500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 43/81] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:49 2022] GET /identity/v3/users/11b1bb7d9ce240aab79c21f829ee58ed => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:49.213706 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:49.213915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.215548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.218897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.228284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 39/82] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:56:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:49.229952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.342853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba047b58-cd88-482a-890e-ea77f9751f06 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.350004 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed754f8-6960-47d9-a31a-cd27bad069b9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.382460 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.383109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.383390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.383795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.385054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.386589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.387842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.388269 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 44/83] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:49 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:49.388475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.388840 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.389083 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.389219 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.407996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:49.427767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:49.491800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.493308 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.509348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c56b7e-1788-4294-8f26-9e48118def77 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.557756 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.558410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.558687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.558963 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.559645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.561157 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.620445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 40/84] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:56:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:56:49.621332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.647108 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 45/85] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:56:49 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:56:49.648227 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.698418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:49.793289 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:49.801564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10660078-0b96-4375-9536-a6d4692381dc None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.814976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5b8d1-b283-4955-bc8d-c1fe5892d42c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:49.857348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.858462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.858950 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.859434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.872376 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:49.873444 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:49.873799 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:49.874046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:49.879044 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.882068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:49.885466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 46/86] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:49 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:49.886842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:49.980429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:49.983561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:50.071518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:50.080323 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d991e-662d-4a4d-9d7d-ec84c28d43b0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:50.143599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:50.144592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:50.145075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:50.146157 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:50.193637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 41/87] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:56:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:56:50.195597 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:50.196695 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:50.284975 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:50.288112 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:50.395679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4db91-1f74-4420-9618-dd8bba44c566 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:50.441315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:50.442298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:50.443461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:50.443981 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:50.445704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:50.448049 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:50.450907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 42/88] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:50 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:50.452444 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:50.473636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:50.499379 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 47/89] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:56:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:56:50.507192 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:50.572226 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232cb25-d95b-43b6-94af-799117c2c305 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:50.615183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:50.616302 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:50.616795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:50.617254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:50.633731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:50.707472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:50.711455 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:50.721109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bd1a6-ad63-44d2-b30f-c5ddd7ca5a88 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:50.748434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:50.769383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:50.770520 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:50.770986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:50.771454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:50.772982 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:50.774948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:50.777480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 48/90] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:50 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:50.779061 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:50.925556 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edd510-a810-4516-9153-fb77bf248f3b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:50.936613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 43/91] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:56:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:56:50.937519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:50.984521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:50.999295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:51.001187 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:51.001187 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:51.001481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:51.002966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=0c11045dcf324aa8bd8417b3e8b91dcb) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:51.003299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:51.005144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:51.012734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 49/92] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:56:50 2022] GET /identity/v3/role_assignments?user.id=0c11045dcf324aa8bd8417b3e8b91dcb&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:51.013628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:51.092424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57089f18-0e45-420b-9cee-c6410734601c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:51.109123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:51.134724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:51.142974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:51.142974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:51.142974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] PATH_INFO: `/v3/projects/cbe8bd7911b145baba0fe7b46324c500` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:51.142974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] RBAC: Authorizing `identity:get_project(project_id=cbe8bd7911b145baba0fe7b46324c500)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:51.142974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:51.142974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 44/93] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:50 2022] GET /identity/v3/projects/cbe8bd7911b145baba0fe7b46324c500 => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:51.143412 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:51.315355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74d128-4c9e-4e7e-826b-31cfb1e264de None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:51.316217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ddffb-11a8-410e-8b52-89feb34e9f37 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:51.355924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:51.356559 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:51.356860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:51.357133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:51.358802 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=11b1bb7d9ce240aab79c21f829ee58ed) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:51.359127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:51.361183 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:51.367344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 50/94] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:56:51 2022] GET /identity/v3/role_assignments?user.id=11b1bb7d9ce240aab79c21f829ee58ed&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:51.373770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:51.414075 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:51.415166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:51.415691 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:51.416350 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:51.423802 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:51.426722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:51.430285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 45/95] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:51 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:51.440926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:51.486201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:51.557829 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8fe5c-c268-4536-bcc4-85d60ae104bf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:51.632123 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e4306-64fd-4154-82e1-baf4b0217fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:51.632681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e4306-64fd-4154-82e1-baf4b0217fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:51.632970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e4306-64fd-4154-82e1-baf4b0217fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:51.633251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e4306-64fd-4154-82e1-baf4b0217fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:51.634025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e4306-64fd-4154-82e1-baf4b0217fa7 None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=cbe8bd7911b145baba0fe7b46324c500, user.id=0c11045dcf324aa8bd8417b3e8b91dcb) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:51.634345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e4306-64fd-4154-82e1-baf4b0217fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:51.635837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e4306-64fd-4154-82e1-baf4b0217fa7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:51.644211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 51/96] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:56:51 2022] GET /identity/v3/role_assignments?user.id=0c11045dcf324aa8bd8417b3e8b91dcb&scope.project.id=cbe8bd7911b145baba0fe7b46324c500&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:51.654174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:51.771111 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072072c-07a6-450b-8f16-d8c9efafbedb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:51.876935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d4290-1e94-4ad3-97e5-06b5f741260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:51.878021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d4290-1e94-4ad3-97e5-06b5f741260b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:51.878500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d4290-1e94-4ad3-97e5-06b5f741260b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:51.878988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d4290-1e94-4ad3-97e5-06b5f741260b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:51.880340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d4290-1e94-4ad3-97e5-06b5f741260b None admin] RBAC: Adding query filter params (role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=0c11045dcf324aa8bd8417b3e8b91dcb) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:51.880908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d4290-1e94-4ad3-97e5-06b5f741260b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:51.883826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d4290-1e94-4ad3-97e5-06b5f741260b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:51.900500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 46/97] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:56:51 2022] GET /identity/v3/role_assignments?user.id=0c11045dcf324aa8bd8417b3e8b91dcb&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 259 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:52.472044 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7694-f3f2-4280-a608-506719bde70d None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:52.473045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7694-f3f2-4280-a608-506719bde70d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:52.473532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e7694-f3f2-4280-a608-506719bde70d None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:52.475247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 52/98] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:56:52.484746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf1b25-8af2-45a9-aadf-82e966c8bfbd None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:52.485392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf1b25-8af2-45a9-aadf-82e966c8bfbd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:52.485841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf1b25-8af2-45a9-aadf-82e966c8bfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:52.547518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-33bf1b25-8af2-45a9-aadf-82e966c8bfbd None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:52.550163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-33bf1b25-8af2-45a9-aadf-82e966c8bfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:52.712139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 47/99] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:53.175656 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a1ad-f55e-40d6-97ef-532b81ece94c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:53.176382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a1ad-f55e-40d6-97ef-532b81ece94c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:53.176876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a1ad-f55e-40d6-97ef-532b81ece94c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:53.178547 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 53/100] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:53.184695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624cf9a-6c87-473a-a317-cfa7dab23896 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:53.185328 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624cf9a-6c87-473a-a317-cfa7dab23896 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:53.185837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624cf9a-6c87-473a-a317-cfa7dab23896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:53.212159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-8624cf9a-6c87-473a-a317-cfa7dab23896 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:53.213798 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-8624cf9a-6c87-473a-a317-cfa7dab23896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:53.222901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a1ad-f55e-40d6-97ef-532b81ece94c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:53.293961 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:53.387675 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 48/101] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:53.482698 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a1ad-f55e-40d6-97ef-532b81ece94c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:53.518177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:53.537011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:53.537711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:53.538002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:53.538287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:53.554024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:53.562686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:53.564956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:53.566577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 54/102] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:56:53.578345 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624cf9a-6c87-473a-a317-cfa7dab23896 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:53.717253 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624cf9a-6c87-473a-a317-cfa7dab23896 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:53.761232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:53.761956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:53.762243 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:53.762523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:53.763218 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:53.763527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:53.764935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:53.777371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 49/103] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:56:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:53.783866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:53.899525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:53.989002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709cb4f-7498-410b-b8b8-30a2ce657909 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:53.994174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:54.074106 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:54.075381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:54.075856 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:54.076317 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] PATH_INFO: `/v3/users/cbef972022e243ab9243e96aaf386c43` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:54.122995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] RBAC: Authorizing `identity:get_user(user_id=cbef972022e243ab9243e96aaf386c43)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:54.127492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:54.130916 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 55/104] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:53 2022] GET /identity/v3/users/cbef972022e243ab9243e96aaf386c43 => generated 289 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:54.141166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:54.221871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7f16-a64c-495f-9639-52d2880d4f7f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:54.295233 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:54.389776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:54.390962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:54.391967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:54.392451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:54.398956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:54.401890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:54.406054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 50/105] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 507 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:54.415440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:54.462860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe219a-4626-430e-af43-9a92e3d3caca None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:54.548899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d077853-6b2d-47e0-94e1-91a2ba9c74cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:54.549823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d077853-6b2d-47e0-94e1-91a2ba9c74cd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:54.550268 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d077853-6b2d-47e0-94e1-91a2ba9c74cd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:54.550701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d077853-6b2d-47e0-94e1-91a2ba9c74cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:54.730788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08a8b7-e7cb-49ae-a5b3-a1949eab5b46 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:54.737396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:54.754946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d077853-6b2d-47e0-94e1-91a2ba9c74cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:54.756913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d077853-6b2d-47e0-94e1-91a2ba9c74cd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:54.801271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:54.821948 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed54bf-3c3d-4c6e-ab48-d0a658ef2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:54.823041 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed54bf-3c3d-4c6e-ab48-d0a658ef2168 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:54.823560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed54bf-3c3d-4c6e-ab48-d0a658ef2168 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:54.824072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed54bf-3c3d-4c6e-ab48-d0a658ef2168 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:54.825317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed54bf-3c3d-4c6e-ab48-d0a658ef2168 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:54.828272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed54bf-3c3d-4c6e-ab48-d0a658ef2168 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:54.883980 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 56/106] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:56:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:56:54.885852 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f27b71-1b39-4688-8bec-604c2fbdf0aa None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:54.886400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f27b71-1b39-4688-8bec-604c2fbdf0aa None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:54.888126 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f27b71-1b39-4688-8bec-604c2fbdf0aa None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:54.889886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 57/107] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:54.895953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f27b71-1b39-4688-8bec-604c2fbdf0aa None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:54.998158 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 51/108] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:56:54 2022] POST /identity/v3/projects => generated 302 bytes in 578 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:56:55.001632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647606-7dac-4ae8-acd1-d61824234a30 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.003598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647606-7dac-4ae8-acd1-d61824234a30 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.006238 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33647606-7dac-4ae8-acd1-d61824234a30 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.011320 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 52/109] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:55.015837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74906fa7-09c5-47b5-99e1-044a0adf2b86 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.017409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74906fa7-09c5-47b5-99e1-044a0adf2b86 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.020728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74906fa7-09c5-47b5-99e1-044a0adf2b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.054469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-74906fa7-09c5-47b5-99e1-044a0adf2b86 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:55.056690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-74906fa7-09c5-47b5-99e1-044a0adf2b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:55.075465 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f27b71-1b39-4688-8bec-604c2fbdf0aa None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:55.191202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 53/110] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:55.193086 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959f6eb-28b8-46ba-a04c-fa72f62378cd None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.193378 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959f6eb-28b8-46ba-a04c-fa72f62378cd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.193681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959f6eb-28b8-46ba-a04c-fa72f62378cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.200605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:55.201268 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.201554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.201860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.204850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:55.208918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:55.210828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 58/111] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:54 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:55.222506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:55.230157 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:55.236991 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-5959f6eb-28b8-46ba-a04c-fa72f62378cd None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:55.238270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-5959f6eb-28b8-46ba-a04c-fa72f62378cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:55.407895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 54/112] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:55.478924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e2806-e811-439d-827f-6c09c34821df None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:55.545364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eabcf3-9b5d-45b2-89fd-8fa6850865c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:55.546488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eabcf3-9b5d-45b2-89fd-8fa6850865c5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.547053 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eabcf3-9b5d-45b2-89fd-8fa6850865c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.548090 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eabcf3-9b5d-45b2-89fd-8fa6850865c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.549492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eabcf3-9b5d-45b2-89fd-8fa6850865c5 None admin] RBAC: Adding query filter params (role.id=79a475679b454999b9d77c6405a7f0a7, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=cbef972022e243ab9243e96aaf386c43) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:55.550059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eabcf3-9b5d-45b2-89fd-8fa6850865c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:55.552834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eabcf3-9b5d-45b2-89fd-8fa6850865c5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:55.563700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 59/113] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:56:55 2022] GET /identity/v3/role_assignments?user.id=cbef972022e243ab9243e96aaf386c43&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=79a475679b454999b9d77c6405a7f0a7 => generated 259 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:55.595298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959f6eb-28b8-46ba-a04c-fa72f62378cd None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:55.636382 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:55.678865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978d490-8c0c-45fc-acf7-ac83f9b4eb8b None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.679058 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978d490-8c0c-45fc-acf7-ac83f9b4eb8b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.679337 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978d490-8c0c-45fc-acf7-ac83f9b4eb8b None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.681118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 60/114] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:55.688857 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f12af5-8337-4114-b407-bfcf85ab6790 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.689155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f12af5-8337-4114-b407-bfcf85ab6790 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.692235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f12af5-8337-4114-b407-bfcf85ab6790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.730878 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-71f12af5-8337-4114-b407-bfcf85ab6790 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:55.732173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-71f12af5-8337-4114-b407-bfcf85ab6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:55.825644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:55.842091 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 61/115] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:55.843723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ec914-ccde-4559-bdaf-a38b6ec415b6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.844296 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ec914-ccde-4559-bdaf-a38b6ec415b6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.844763 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ec914-ccde-4559-bdaf-a38b6ec415b6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.846421 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 62/116] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:55.854542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959f6eb-28b8-46ba-a04c-fa72f62378cd None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:55.862892 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4a60d-dc76-404f-a87b-faa10f525980 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.863526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4a60d-dc76-404f-a87b-faa10f525980 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.864018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4a60d-dc76-404f-a87b-faa10f525980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.910795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-73c4a60d-dc76-404f-a87b-faa10f525980 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:55.912385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-73c4a60d-dc76-404f-a87b-faa10f525980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:55.930214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:55.930214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:55.930214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:55.930214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:55.943753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:55.956358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:55.957301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:56:55.958109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 55/117] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:56:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:56:55.966037 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.114026 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 63/118] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:56.114942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4a60d-dc76-404f-a87b-faa10f525980 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.128797 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:56.204811 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb0795-a947-48fb-8e5d-b48d93705748 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:56.306059 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:56.307182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:56.307700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:56.308358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:56.309937 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:56.310504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:56.316765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4a60d-dc76-404f-a87b-faa10f525980 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:56.319389 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:56.349472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 56/119] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:56:56.353812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.361395 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:56.387807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:56.388866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:56.389392 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:56.390043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:56.422025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:56.427751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:56.428688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:56.429462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:56.439609 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 64/120] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:56.440948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.493027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db4319-a9dc-4349-a816-3b57f9591c84 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:56.590118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6fad2-011b-4354-96bb-ef2898cef1bd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:56.607281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:56.608096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:56.608474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:56.608886 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:56.617826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:56.624195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:56.630816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:56.633958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:56:56.634795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 57/121] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:56.635446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:56.635446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:56.635585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:56.636124 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] PATH_INFO: `/v3/users/11b1bb7d9ce240aab79c21f829ee58ed` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:56.636750 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.636966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] RBAC: Authorizing `identity:get_user(user_id=11b1bb7d9ce240aab79c21f829ee58ed)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:56.638518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:56.640213 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 65/122] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/users/11b1bb7d9ce240aab79c21f829ee58ed => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:56.641862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.722841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:56.755741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f62a1-363d-43f0-b71b-461883c8ddbc None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:56.786184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf94be-7eb6-40bb-94a7-b1dae56cf436 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:56.795490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:56.796723 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:56.797015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:56.797334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:56.798070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:56.798453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:56.800209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:56.812038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 58/123] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:56.812750 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.840623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:56.841772 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:56.842281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:56.842825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:56.855859 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:56.861475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:56.862410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:56:56.863199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:56.865880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 66/124] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:56.867200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.881371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:56.926925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4376d-ee13-4680-8109-93a4a309b03e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:56.965803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:56.966641 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:56.967080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:56.967383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:56.968229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:56.968642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:56.970260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:56.979696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 59/125] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:56.984806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:56.998096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ee3d-428e-4a28-98cf-37824696c431 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.015617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:57.040858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.041807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.042251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.042686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.044169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.045855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.048528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 67/126] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:57.049595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.099042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc464de-4b2b-487e-ad27-53f9c8b2c3b3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.124442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:57.138199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.138199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.138515 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.138850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.140282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.142467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.144266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 60/127] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:56 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:57.145211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.185714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccd5a8-c8e2-4a83-a0af-42abd7c90611 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.232423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.233288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.233828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.234255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.235111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:57.235560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.237321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.247791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 68/128] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:56:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:57.248960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.263032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8856b36-af88-4566-97e2-9b9658d074a7 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.303399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.304077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.304440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.304802 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.306060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:57.306174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.307744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.313925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 61/129] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:57 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:57.313925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.380736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb5892-32b9-4a4c-a272-ca6327124211 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.387912 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:57.422305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.424426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.425547 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.426062 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/11b1bb7d9ce240aab79c21f829ee58ed/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.428593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba34f3047cae4d5d8da4129e3b26f8b3, user_id=11b1bb7d9ce240aab79c21f829ee58ed, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.430415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.434573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b633f43-c7c6-42ca-983b-bff0aa954878 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.475542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.476203 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.476483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.476755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] PATH_INFO: `/v3/projects/cbe8bd7911b145baba0fe7b46324c500` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.478015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] RBAC: Authorizing `identity:get_project(project_id=cbe8bd7911b145baba0fe7b46324c500)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.479461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.481355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 62/130] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:57 2022] GET /identity/v3/projects/cbe8bd7911b145baba0fe7b46324c500 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:57.482007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.509115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 69/131] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:56:57 2022] PUT /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/11b1bb7d9ce240aab79c21f829ee58ed/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:56:57.519065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.625488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362c031-219e-43c2-a466-5eeed058bfb3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.665554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.666292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.666575 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.666855 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.668247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.669842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.672121 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 63/132] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:57 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:57.673373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.678274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:57.796580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b78da-1112-44c3-8a18-ad1cd6f5f470 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.801801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:57.812940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99e7c6-9c5b-4719-9824-c460144030fe None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:57.854333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.855162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.855453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.855736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.856906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.859409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.861080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 70/133] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:57 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:57.861971 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:57.864551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a984a9-7f03-4c32-aaf1-9f44fd833bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:57.865763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a984a9-7f03-4c32-aaf1-9f44fd833bcb None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.866250 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a984a9-7f03-4c32-aaf1-9f44fd833bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.866739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a984a9-7f03-4c32-aaf1-9f44fd833bcb None admin] PATH_INFO: `/v3/projects/cbe8bd7911b145baba0fe7b46324c500/users/0c11045dcf324aa8bd8417b3e8b91dcb/roles/bddb19f21b7f472589b9154cb2aeeecc` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.870281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a984a9-7f03-4c32-aaf1-9f44fd833bcb None admin] RBAC: Authorizing `identity:create_grant(project_id=cbe8bd7911b145baba0fe7b46324c500, user_id=0c11045dcf324aa8bd8417b3e8b91dcb, role_id=bddb19f21b7f472589b9154cb2aeeecc)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:57.873016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a984a9-7f03-4c32-aaf1-9f44fd833bcb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:57.987968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 64/134] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:56:57 2022] PUT /identity/v3/projects/cbe8bd7911b145baba0fe7b46324c500/users/0c11045dcf324aa8bd8417b3e8b91dcb/roles/bddb19f21b7f472589b9154cb2aeeecc => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:56:57.990383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cec41-2668-4f26-87bb-10071d25033b None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:57.990991 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cec41-2668-4f26-87bb-10071d25033b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:57.991544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cec41-2668-4f26-87bb-10071d25033b None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:57.993408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 65/135] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:57.994674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cec41-2668-4f26-87bb-10071d25033b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:58.081619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ee4f5-033b-4ab7-a3ac-7bcd65ec87b2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:58.134770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:58.166366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23438f28-7439-4dac-b2fe-b4aa2dec4575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:58.168034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23438f28-7439-4dac-b2fe-b4aa2dec4575 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:58.168434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23438f28-7439-4dac-b2fe-b4aa2dec4575 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:58.168838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23438f28-7439-4dac-b2fe-b4aa2dec4575 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:58.170351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23438f28-7439-4dac-b2fe-b4aa2dec4575 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:58.172081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23438f28-7439-4dac-b2fe-b4aa2dec4575 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:58.174236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 71/136] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:57 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:58.175542 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac948db-d26c-4615-8edd-cec60bd4335e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:58.175835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac948db-d26c-4615-8edd-cec60bd4335e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:58.176114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac948db-d26c-4615-8edd-cec60bd4335e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:58.208530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6ac948db-d26c-4615-8edd-cec60bd4335e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:58.209942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6ac948db-d26c-4615-8edd-cec60bd4335e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:58.254649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:58.265186 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cec41-2668-4f26-87bb-10071d25033b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:58.298195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 72/137] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:56:58.299273 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac948db-d26c-4615-8edd-cec60bd4335e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:58.331687 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dcada-758b-4ed4-b69f-868048cf0ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:58.333237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dcada-758b-4ed4-b69f-868048cf0ee2 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:58.333962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dcada-758b-4ed4-b69f-868048cf0ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:58.334631 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dcada-758b-4ed4-b69f-868048cf0ee2 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/0c11045dcf324aa8bd8417b3e8b91dcb/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:58.341764 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dcada-758b-4ed4-b69f-868048cf0ee2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba34f3047cae4d5d8da4129e3b26f8b3, user_id=0c11045dcf324aa8bd8417b3e8b91dcb, role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:58.345130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dcada-758b-4ed4-b69f-868048cf0ee2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:58.429638 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 66/138] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:56:57 2022] PUT /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/0c11045dcf324aa8bd8417b3e8b91dcb/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:56:58.433425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac948db-d26c-4615-8edd-cec60bd4335e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:58.487876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:58.488599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:58.488889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:58.489173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/0c11045dcf324aa8bd8417b3e8b91dcb/roles/bddb19f21b7f472589b9154cb2aeeecc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:58.491580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba34f3047cae4d5d8da4129e3b26f8b3, user_id=0c11045dcf324aa8bd8417b3e8b91dcb, role_id=bddb19f21b7f472589b9154cb2aeeecc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:58.493083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:58.546834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fa42-7edd-4e2f-aa8b-c0ce70e9f96d None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:58.548329 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fa42-7edd-4e2f-aa8b-c0ce70e9f96d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:58.549105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fa42-7edd-4e2f-aa8b-c0ce70e9f96d None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:58.553847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 67/139] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:56:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:56:58.560380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87404e6-467c-4bc3-87d7-07cf3e97223b None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:58.561255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87404e6-467c-4bc3-87d7-07cf3e97223b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:58.562058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87404e6-467c-4bc3-87d7-07cf3e97223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:58.589652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 73/140] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:56:58 2022] PUT /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/0c11045dcf324aa8bd8417b3e8b91dcb/roles/bddb19f21b7f472589b9154cb2aeeecc => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:56:58.611829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-e87404e6-467c-4bc3-87d7-07cf3e97223b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:56:58.614491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-e87404e6-467c-4bc3-87d7-07cf3e97223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:56:58.768590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 68/141] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:56:58.772497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:58.830454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:58.932204 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29bf9-d364-47dd-89c8-03305fe594c8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:58.979862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:58.980528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:58.980803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:58.981062 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:58.982253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:58.983722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:58.985368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 74/142] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:56:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:58.991431 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87404e6-467c-4bc3-87d7-07cf3e97223b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:59.065067 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:59.143824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87404e6-467c-4bc3-87d7-07cf3e97223b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:59.174095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:59.201041 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:59.201696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:59.201978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:59.202251 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:59.202917 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:59.204941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:59.320401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:59.334587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 69/143] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:56:58 2022] POST /identity/v3/users => generated 283 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:56:59.335657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae693b2-bf55-4d48-84b3-3bacb7c71b63 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:59.393269 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:59.394065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:59.394316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:59.394651 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:59.410339 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:56:59.415591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:56:59.417553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:56:59.418808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 75/144] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:56:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:56:59.423766 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:59.550942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:56:59.555769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f932558-2d5d-4511-a5b4-5bc8031fa546 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:59.611253 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:59.611904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:59.612188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:59.612468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:59.613183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:56:59.613639 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:59.615305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:59.631733 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 70/145] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:56:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:59.637657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:59.760285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e2f1-0f42-4efd-ad51-151fcb92ad59 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:56:59.814465 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:56:59.815169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:56:59.815489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:56:59.815767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] PATH_INFO: `/v3/users/cbef972022e243ab9243e96aaf386c43` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:56:59.817061 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cbef972022e243ab9243e96aaf386c43)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:56:59.818741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:56:59.821142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 76/146] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:56:59 2022] GET /identity/v3/users/cbef972022e243ab9243e96aaf386c43 => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:56:59.827215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:56:59.967756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d7fe9-656d-41c2-b650-ebab32f58a5c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:00.014844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff433f-1e8f-449d-9567-81088b1b825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:00.015943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff433f-1e8f-449d-9567-81088b1b825a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:00.015943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff433f-1e8f-449d-9567-81088b1b825a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:00.016915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff433f-1e8f-449d-9567-81088b1b825a None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:00.017799 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff433f-1e8f-449d-9567-81088b1b825a None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:00.019252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff433f-1e8f-449d-9567-81088b1b825a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:00.020920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 71/147] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:56:59 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:00.026812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:00.065477 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:00.202699 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa608d7-89dd-4241-8a01-6adddcd179d6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:00.276766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc410aad-a3d0-4f04-9f6f-a720209f6803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:00.277819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc410aad-a3d0-4f04-9f6f-a720209f6803 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:00.278287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc410aad-a3d0-4f04-9f6f-a720209f6803 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:00.278756 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc410aad-a3d0-4f04-9f6f-a720209f6803 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/cbef972022e243ab9243e96aaf386c43/roles/79a475679b454999b9d77c6405a7f0a7` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:00.282890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc410aad-a3d0-4f04-9f6f-a720209f6803 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba34f3047cae4d5d8da4129e3b26f8b3, user_id=cbef972022e243ab9243e96aaf386c43, role_id=79a475679b454999b9d77c6405a7f0a7)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:00.285521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc410aad-a3d0-4f04-9f6f-a720209f6803 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:00.368798 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 77/148] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:57:00 2022] PUT /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/users/cbef972022e243ab9243e96aaf386c43/roles/79a475679b454999b9d77c6405a7f0a7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:57:00.894547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df15e7-9c1d-4ef9-be5d-2754e986df74 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:00.894925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df15e7-9c1d-4ef9-be5d-2754e986df74 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:00.895090 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df15e7-9c1d-4ef9-be5d-2754e986df74 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:00.896686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 72/149] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:00.902226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47cbb3-c54c-4a58-9d69-3cd0a2b13489 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:00.902519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47cbb3-c54c-4a58-9d69-3cd0a2b13489 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:00.905445 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47cbb3-c54c-4a58-9d69-3cd0a2b13489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:00.930043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6a47cbb3-c54c-4a58-9d69-3cd0a2b13489 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:00.931523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6a47cbb3-c54c-4a58-9d69-3cd0a2b13489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:01.025179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 78/150] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:01.176227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db690-d7f6-48b1-8671-ab03ad67fcd3 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:01.177095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db690-d7f6-48b1-8671-ab03ad67fcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:01.177509 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db690-d7f6-48b1-8671-ab03ad67fcd3 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:01.179398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 73/151] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:01.185952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fb6af-3f3f-49e6-bcf1-83d58178f87b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:01.186482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fb6af-3f3f-49e6-bcf1-83d58178f87b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:01.186482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fb6af-3f3f-49e6-bcf1-83d58178f87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:01.212607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a79fb6af-3f3f-49e6-bcf1-83d58178f87b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:01.213881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a79fb6af-3f3f-49e6-bcf1-83d58178f87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:01.228599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:01.307430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 79/152] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:01.788489 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437db690-d7f6-48b1-8671-ab03ad67fcd3 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:01.831020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:01.935485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437db690-d7f6-48b1-8671-ab03ad67fcd3 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:01.951175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3b133-0de0-4007-975f-4b57ef707426 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:01.951819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3b133-0de0-4007-975f-4b57ef707426 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:01.952302 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3b133-0de0-4007-975f-4b57ef707426 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:01.954534 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 80/153] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:01.971555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57467967-4b7c-4be5-9bfd-e290d726df69 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:01.972202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57467967-4b7c-4be5-9bfd-e290d726df69 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:01.973427 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57467967-4b7c-4be5-9bfd-e290d726df69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:01.994857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:01.995939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:01.996412 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:01.996887 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:02.019967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-57467967-4b7c-4be5-9bfd-e290d726df69 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:02.020976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:02.022044 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-57467967-4b7c-4be5-9bfd-e290d726df69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:02.026216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:02.027120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:02.027904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 74/154] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:02.028714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:02.225223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 81/155] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:02.227653 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57467967-4b7c-4be5-9bfd-e290d726df69 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:02.266970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:02.282672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c880f-eca7-417f-9f67-412e30f80d95 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:02.351984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:02.353070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:02.353555 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:02.354147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:02.379513 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:02.394643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:02.395651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:02.396424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:02.402026 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 75/156] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:57:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:02.410698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:02.466109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:02.526832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57467967-4b7c-4be5-9bfd-e290d726df69 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:02.617643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bec60-b24f-4858-b478-bd103e48b159 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:02.638940 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:02.640520 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:02.641354 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:02.642149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:02.643621 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:02.644515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:02.647478 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:02.668111 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:02.668843 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:02.669134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:02.669414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:02.670147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:02.670458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:02.671877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:02.676367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 82/157] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:02.682531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:02.689593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 76/158] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:02.695936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:02.722215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:02.849989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf5492-4f7a-4d46-a73c-ba5daf3e788a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:02.903498 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:02.904402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:02.904848 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:02.905326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] PATH_INFO: `/v3/users/11b1bb7d9ce240aab79c21f829ee58ed` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:02.907441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] RBAC: Authorizing `identity:get_user(user_id=11b1bb7d9ce240aab79c21f829ee58ed)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:02.909063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:02.911242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 77/159] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:02 2022] GET /identity/v3/users/11b1bb7d9ce240aab79c21f829ee58ed => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:02.916991 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:02.947723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f880eca-e686-4e9b-9a4e-38a9bc80dedd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:03.043041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba23b7-8a29-48b6-948e-232adf222471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:03.044585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba23b7-8a29-48b6-948e-232adf222471 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:03.045330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba23b7-8a29-48b6-948e-232adf222471 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:03.046176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba23b7-8a29-48b6-948e-232adf222471 None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:03.049133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba23b7-8a29-48b6-948e-232adf222471 None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:03.049936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:03.056555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba23b7-8a29-48b6-948e-232adf222471 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:03.059907 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 83/160] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:02 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:03.062222 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8713da-1683-4d60-b989-c6ad82228098 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:03.062840 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8713da-1683-4d60-b989-c6ad82228098 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:03.068693 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8713da-1683-4d60-b989-c6ad82228098 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:03.070401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 84/161] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:03.072162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8713da-1683-4d60-b989-c6ad82228098 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:03.079384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bd810-7b20-4815-ae36-bd8bb5c0b66e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:03.095757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:03.203076 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:03.204574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:03.205058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:03.205529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:03.314798 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8713da-1683-4d60-b989-c6ad82228098 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:03.420338 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367d809-ec57-4811-bc60-ebd75b3e2510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:03.421004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367d809-ec57-4811-bc60-ebd75b3e2510 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:03.421285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367d809-ec57-4811-bc60-ebd75b3e2510 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:03.421554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367d809-ec57-4811-bc60-ebd75b3e2510 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:03.456430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367d809-ec57-4811-bc60-ebd75b3e2510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4367d809-ec57-4811-bc60-ebd75b3e2510 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:03.462314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:03.463189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:03.464054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 85/162] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:03.471144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea03f3b-4e7f-42ad-9a9b-225f60e0ea22 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:03.471435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea03f3b-4e7f-42ad-9a9b-225f60e0ea22 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:03.471709 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea03f3b-4e7f-42ad-9a9b-225f60e0ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:03.510213 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6ea03f3b-4e7f-42ad-9a9b-225f60e0ea22 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:03.510954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:03.511961 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6ea03f3b-4e7f-42ad-9a9b-225f60e0ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:03.514780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:03.636130 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:03.647124 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 86/163] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:03.648287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea03f3b-4e7f-42ad-9a9b-225f60e0ea22 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:03.827522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:03.858751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea03f3b-4e7f-42ad-9a9b-225f60e0ea22 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:03.907323 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 78/164] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 991 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:57:03.911650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:03.947136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:03.947828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:03.948334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:03.948760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.078919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:04.098786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:04.100589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:04.265478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db5cdc-8f32-4cd3-b47f-49ae140d7106 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:04.283564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 87/165] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:04.284748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:04.323101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:04.324117 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.324578 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.325225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.326564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:04.327272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:04.329477 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:04.342529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 79/166] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:04.344014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:04.398368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:04.404487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e84bc-c1e0-4c82-818e-be00e1c84cec None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:04.405686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:04.474737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67810b76-5203-4ed6-ba49-cc0c322f3d87 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:04.478394 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:04.479071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.479355 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.479634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.486494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:04.488014 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:04.489865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 88/167] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:04 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:04.490667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:04.521119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20179d2f-5933-4644-a9e9-50b20a27ad4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:04.522011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179d2f-5933-4644-a9e9-50b20a27ad4f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.522142 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179d2f-5933-4644-a9e9-50b20a27ad4f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.522400 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179d2f-5933-4644-a9e9-50b20a27ad4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.534922 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20179d2f-5933-4644-a9e9-50b20a27ad4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-20179d2f-5933-4644-a9e9-50b20a27ad4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:04.540834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:04.542658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 80/168] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:04.544210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b96821-7b61-4e43-aad1-20a492e946a9 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.550794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b96821-7b61-4e43-aad1-20a492e946a9 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.550794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b96821-7b61-4e43-aad1-20a492e946a9 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.550794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 81/169] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:04.551821 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b96821-7b61-4e43-aad1-20a492e946a9 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:04.674765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b96821-7b61-4e43-aad1-20a492e946a9 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:04.676551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:04.716182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:04.717167 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.717645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.717749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a028-30ac-4466-a562-d546085ba980 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:04.718191 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.719796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:04.721632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:04.723894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 82/170] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:04 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:04.730955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:04.800030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:04.800809 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.801094 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.801483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] PATH_INFO: `/v3/projects/cbe8bd7911b145baba0fe7b46324c500` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.806660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] RBAC: Authorizing `identity:get_project(project_id=cbe8bd7911b145baba0fe7b46324c500)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:04.808236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:04.810081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 89/171] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:04 2022] GET /identity/v3/projects/cbe8bd7911b145baba0fe7b46324c500 => generated 312 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:04.811067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:04.857763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ea59c-c9ca-474a-aa96-5e4d606db4b9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:04.878804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:04.901748 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf8cba-ae55-48fd-9f21-0e5e55c6bf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:04.902741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8cba-ae55-48fd-9f21-0e5e55c6bf1b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.903254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8cba-ae55-48fd-9f21-0e5e55c6bf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.903796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf8cba-ae55-48fd-9f21-0e5e55c6bf1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.904863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf8cba-ae55-48fd-9f21-0e5e55c6bf1b None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=11b1bb7d9ce240aab79c21f829ee58ed) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:04.905385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf8cba-ae55-48fd-9f21-0e5e55c6bf1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:04.907784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf8cba-ae55-48fd-9f21-0e5e55c6bf1b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:04.916438 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 83/172] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:04 2022] GET /identity/v3/role_assignments?user.id=11b1bb7d9ce240aab79c21f829ee58ed&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:04.925073 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a358-cffa-4b30-8244-07a2ccf74378 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:04.925711 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a358-cffa-4b30-8244-07a2ccf74378 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:04.926189 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382a358-cffa-4b30-8244-07a2ccf74378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:04.966970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-9382a358-cffa-4b30-8244-07a2ccf74378 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:04.968560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9382a358-cffa-4b30-8244-07a2ccf74378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:05.016870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:05.052280 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d10343-cc8e-4028-a7b0-c36d5fa5f4b6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:05.104411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:05.105075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.105350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.105632 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.106346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:05.106646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:05.108074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:05.118016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 90/173] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:05.118957 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:05.123412 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 84/174] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:05.124927 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382a358-cffa-4b30-8244-07a2ccf74378 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:05.263140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:05.271201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827a2d9-4e58-4972-ad52-f2dbd70541ab None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:05.310928 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e554331-c292-4848-8930-5184db182c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:05.311357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e554331-c292-4848-8930-5184db182c23 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.311599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e554331-c292-4848-8930-5184db182c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.311867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e554331-c292-4848-8930-5184db182c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.317988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382a358-cffa-4b30-8244-07a2ccf74378 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:05.395151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e554331-c292-4848-8930-5184db182c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:05.396704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e554331-c292-4848-8930-5184db182c23 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:05.404135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb454305-4a3b-4336-b317-138b3f4f718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:05.405719 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb454305-4a3b-4336-b317-138b3f4f718b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.406482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb454305-4a3b-4336-b317-138b3f4f718b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.407740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb454305-4a3b-4336-b317-138b3f4f718b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.409575 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb454305-4a3b-4336-b317-138b3f4f718b None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=cbe8bd7911b145baba0fe7b46324c500, user.id=0c11045dcf324aa8bd8417b3e8b91dcb) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:05.410598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb454305-4a3b-4336-b317-138b3f4f718b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:05.414103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb454305-4a3b-4336-b317-138b3f4f718b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:05.426361 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 85/175] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:05 2022] GET /identity/v3/role_assignments?user.id=0c11045dcf324aa8bd8417b3e8b91dcb&scope.project.id=cbe8bd7911b145baba0fe7b46324c500&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:05.429125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afc6b2-ff6d-44e8-95ef-d2636a035c9b None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.429972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afc6b2-ff6d-44e8-95ef-d2636a035c9b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.430655 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afc6b2-ff6d-44e8-95ef-d2636a035c9b None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.433776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 86/176] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:05.436486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a639c3-fd10-4612-a89b-53a903aaea67 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.437384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a639c3-fd10-4612-a89b-53a903aaea67 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.437951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a639c3-fd10-4612-a89b-53a903aaea67 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.440467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 87/177] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:05.441899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a639c3-fd10-4612-a89b-53a903aaea67 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:05.517104 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:05.523532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:05.532703 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 91/178] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:05.538153 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635581f9-a3d4-4aae-abcf-1423053afdbf None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.538823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635581f9-a3d4-4aae-abcf-1423053afdbf None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.539937 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635581f9-a3d4-4aae-abcf-1423053afdbf None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.541702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 92/179] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:05.542759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c34fac-04a9-48b6-8bfb-25e98d62d61f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.543043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c34fac-04a9-48b6-8bfb-25e98d62d61f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.543325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c34fac-04a9-48b6-8bfb-25e98d62d61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.567359 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-69c34fac-04a9-48b6-8bfb-25e98d62d61f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:05.568649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-69c34fac-04a9-48b6-8bfb-25e98d62d61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:05.611551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a639c3-fd10-4612-a89b-53a903aaea67 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:05.650879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:05.651544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.651805 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.652079 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] PATH_INFO: `/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.653239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] RBAC: Authorizing `identity:get_user(user_id=0c11045dcf324aa8bd8417b3e8b91dcb)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:05.654049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 93/180] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:05.654839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:05.662603 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 88/181] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:05 2022] GET /identity/v3/users/0c11045dcf324aa8bd8417b3e8b91dcb => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:05.662603 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:05.663389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278081d3-2510-4bdc-a743-2f7a9d1009b9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.663674 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278081d3-2510-4bdc-a743-2f7a9d1009b9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.663949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278081d3-2510-4bdc-a743-2f7a9d1009b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.685912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-278081d3-2510-4bdc-a743-2f7a9d1009b9 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:05.687158 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-278081d3-2510-4bdc-a743-2f7a9d1009b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:05.796705 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 94/182] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:05.798569 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278081d3-2510-4bdc-a743-2f7a9d1009b9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:05.815454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:05.823542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb32c56-8621-4023-9bfe-5d32ee3075e4 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:05.866543 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:05.874529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd043494-9c32-47e7-8eec-e47058ce58e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:05.876289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043494-9c32-47e7-8eec-e47058ce58e6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.876717 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043494-9c32-47e7-8eec-e47058ce58e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.877188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043494-9c32-47e7-8eec-e47058ce58e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.893688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd043494-9c32-47e7-8eec-e47058ce58e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-fd043494-9c32-47e7-8eec-e47058ce58e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:05.902014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:05.903123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:05.904096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:05.906217 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 89/183] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:57:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:05.911968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285af84-27a0-4f83-b408-c397b792214b None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.911968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285af84-27a0-4f83-b408-c397b792214b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.911968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285af84-27a0-4f83-b408-c397b792214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.926873 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278081d3-2510-4bdc-a743-2f7a9d1009b9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:05.946499 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2285af84-27a0-4f83-b408-c397b792214b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:05.948593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2285af84-27a0-4f83-b408-c397b792214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:05.973985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:05.974639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:05.974922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:05.975198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:05.976792 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:05.978618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:05.985564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 95/184] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:05 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:05.985564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.045643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 90/185] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:06.051487 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285af84-27a0-4f83-b408-c397b792214b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.156023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90add-ad11-482c-b96b-0ace6bb10400 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:06.177692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:06.181761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:06.194607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:06.195530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:06.195753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:06.199155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:06.203984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285af84-27a0-4f83-b408-c397b792214b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:06.259177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:06.260372 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:06.260890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:06.261383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:06.262680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:06.263224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:06.265740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:06.282962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 91/186] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:06.284183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.291766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:06.293256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:06.402929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 96/187] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:06.404268 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.427870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e1ff5-6e25-4ee6-85b2-32bb610e229a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:06.434355 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:06.467581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:06.468332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:06.468656 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:06.468952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:06.480899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:06.486650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:06.488355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 92/188] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:06.489293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.502942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:06.530237 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934d81-cefe-42ea-a2b1-f6da6020993d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:06.569168 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:06.570137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:06.570414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:06.570705 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:06.571433 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] RBAC: Adding query filter params (role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=0c11045dcf324aa8bd8417b3e8b91dcb) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:06.571739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:06.573267 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:06.579608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 97/189] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/role_assignments?user.id=0c11045dcf324aa8bd8417b3e8b91dcb&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:06.580549 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.622506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b19fb6-b809-4664-a079-723064e58566 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:06.691709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:06.692778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:06.693232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:06.693755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:06.710679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:06.714469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:06.725925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:06.730233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:06.731824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 93/190] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:06.738395 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0dfeb-85e4-4bfc-a6d0-441252bda2f5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:06.738730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.785729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:06.786525 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:06.787516 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:06.787516 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] PATH_INFO: `/v3/users/a6f6e9287c18489c84cbb267d909635b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:06.808757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] RBAC: Authorizing `identity:get_user(user_id=a6f6e9287c18489c84cbb267d909635b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:06.810808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:06.813887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 98/191] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/users/a6f6e9287c18489c84cbb267d909635b => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:06.815214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.869729 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:06.919026 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb42f0-b34f-4f73-8849-598eb468708b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:06.961384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:06.961975 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:06.962259 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:06.962532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:06.963693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:06.965151 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:06.966776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 94/192] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:06.969845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:06.971179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcbb56-0175-475f-a46b-440616914b72 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.012015 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.013009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.013464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.013879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.015470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.017550 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.019563 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 99/193] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:07.020044 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:07.045695 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:07.093018 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84aae3-f314-42fb-8a60-a3aeeadbc3e1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.131461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.132084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.132367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.132645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.133317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:07.133661 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.135054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.137678 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:07.145316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 95/194] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:07.147002 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:07.182267 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dac5da-003f-4eb4-a0a4-555881dafaaf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.231989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.232961 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.233325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.233761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.235031 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:07.235546 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.238039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.250711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 100/195] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:07.251787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:07.290573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805d99d-89cf-4ecf-8e0a-d4bb10de7111 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.343001 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.343730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.344129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.344485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.348154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:07.377196 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b885155-d457-4a5b-b2cd-82cadaecba01 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.416518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.417137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.417420 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.417728 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.418406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.419909 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.433504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:07.437319 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.438982 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.536562 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 101/196] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:07 2022] POST /identity/v3/users => generated 297 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:07.538100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:07.555242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 96/197] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:07.556440 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:07.689896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4578cd-ec29-498e-b3c9-f11d8bd76b34 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.707335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:07.710722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80729d74-0875-473f-8111-90a9ce16b858 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.732402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:07.740503 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.740870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.741156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.742248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=0c11045dcf324aa8bd8417b3e8b91dcb) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:07.742559 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.744073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.751127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 102/198] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:07 2022] GET /identity/v3/role_assignments?user.id=0c11045dcf324aa8bd8417b3e8b91dcb&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:07.751980 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:07.759025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.759692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.759978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.760284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] PATH_INFO: `/v3/users/cbef972022e243ab9243e96aaf386c43` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.761680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cbef972022e243ab9243e96aaf386c43)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.763276 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.765579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 97/199] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:07 2022] GET /identity/v3/users/cbef972022e243ab9243e96aaf386c43 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:07.771119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:07.915137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbef916-70e8-4269-98a9-f13a1f078daa None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.952373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430d117-c942-4fcc-9a3c-3b33826f73b8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:07.967473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:07.969372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:07.969372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:07.969818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] PATH_INFO: `/v3/users/a6f6e9287c18489c84cbb267d909635b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:07.971461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] RBAC: Authorizing `identity:get_user(user_id=a6f6e9287c18489c84cbb267d909635b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:07.973781 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:07.976453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 103/200] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:07 2022] GET /identity/v3/users/a6f6e9287c18489c84cbb267d909635b => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:07.977313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:08.004964 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:08.005742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:08.006045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:08.006319 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:08.008113 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:08.009586 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:08.014164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 98/201] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:07 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:08.014164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:08.051008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:08.067352 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:08.120210 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac89ea-2226-4576-bcf2-81dc84af7056 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:08.153040 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d9fa-3360-45eb-8133-26cf9b91026f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:08.173514 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:08.174214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:08.174502 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:08.174778 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:08.206512 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:08.207782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:08.208291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:08.208781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:08.307412 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:08.309571 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:08.310853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:08.314645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:08.355893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:08.430332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:08.448960 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 99/202] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:08.448960 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:08.495531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 104/203] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:08.496435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:08.673194 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acf4e1-8892-423b-a744-08769e3ea246 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:08.676658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3a84-3bc6-4b29-a386-a1dd0b046628 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:08.745903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:08.747127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:08.747858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:08.748481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:08.750105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:08.752044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:08.753232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:08.753489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:08.754040 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:08.754844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:08.756697 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=a6f6e9287c18489c84cbb267d909635b) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:08.757431 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:08.757912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 105/204] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:08 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:08.759626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:08.760857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:08.765487 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:08.780485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 100/205] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:08 2022] GET /identity/v3/role_assignments?user.id=a6f6e9287c18489c84cbb267d909635b&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:08.783255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:08.838024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:08.988048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58045a3b-66fd-4821-960a-2a9b582d8835 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:09.065186 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:09.066312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:09.066815 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:09.067286 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:09.069639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:09.074295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:09.081640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 106/206] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:08 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:09.088472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:09.159813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff842bd-2367-424e-99ac-a9899801e81e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:09.225399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fc9a8-e456-459a-96b6-0a3bdfcc40e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:09.226098 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc9a8-e456-459a-96b6-0a3bdfcc40e6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:09.226371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc9a8-e456-459a-96b6-0a3bdfcc40e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:09.226644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc9a8-e456-459a-96b6-0a3bdfcc40e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:09.227511 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc9a8-e456-459a-96b6-0a3bdfcc40e6 None admin] RBAC: Adding query filter params (role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=a6f6e9287c18489c84cbb267d909635b) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:09.227665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc9a8-e456-459a-96b6-0a3bdfcc40e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:09.229132 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc9a8-e456-459a-96b6-0a3bdfcc40e6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:09.237231 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 101/207] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:08 2022] GET /identity/v3/role_assignments?user.id=a6f6e9287c18489c84cbb267d909635b&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 259 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:09.319053 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:09.327585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892eb44-6706-4683-992d-b3ec50a225c6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:09.410596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df96584-d1cd-4212-a229-731c14bb934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:09.412080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df96584-d1cd-4212-a229-731c14bb934a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:09.412620 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df96584-d1cd-4212-a229-731c14bb934a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:09.413216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df96584-d1cd-4212-a229-731c14bb934a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:09.414740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df96584-d1cd-4212-a229-731c14bb934a None admin] RBAC: Adding query filter params (role.id=79a475679b454999b9d77c6405a7f0a7, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3, user.id=cbef972022e243ab9243e96aaf386c43) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:09.415553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df96584-d1cd-4212-a229-731c14bb934a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:09.418533 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df96584-d1cd-4212-a229-731c14bb934a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:09.438892 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 107/208] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:09 2022] GET /identity/v3/role_assignments?user.id=cbef972022e243ab9243e96aaf386c43&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=79a475679b454999b9d77c6405a7f0a7 => generated 608 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:10.454368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca851f8-726e-460b-82e1-796d599d3c26 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:10.455125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca851f8-726e-460b-82e1-796d599d3c26 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:10.455125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca851f8-726e-460b-82e1-796d599d3c26 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:10.456620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 102/209] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:10.464280 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85697c03-6821-407a-beb7-6b74f99afeab None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:10.464770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85697c03-6821-407a-beb7-6b74f99afeab None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:10.465230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85697c03-6821-407a-beb7-6b74f99afeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:10.493986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-85697c03-6821-407a-beb7-6b74f99afeab None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:10.496186 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-85697c03-6821-407a-beb7-6b74f99afeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:10.562100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fecbc-7a94-43ca-8876-66fb6cebe2b0 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:10.562647 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fecbc-7a94-43ca-8876-66fb6cebe2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:10.563158 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fecbc-7a94-43ca-8876-66fb6cebe2b0 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:10.565964 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 103/210] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:10.575241 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7d87a-e03a-40d1-8056-0111bf464c29 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:10.575840 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7d87a-e03a-40d1-8056-0111bf464c29 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:10.576327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7d87a-e03a-40d1-8056-0111bf464c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:10.611474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2af7d87a-e03a-40d1-8056-0111bf464c29 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:10.613994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2af7d87a-e03a-40d1-8056-0111bf464c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:10.631117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 108/211] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:10.729366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:10.745206 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 104/212] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:10.903855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85697c03-6821-407a-beb7-6b74f99afeab None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:11.082283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:11.097575 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7d87a-e03a-40d1-8056-0111bf464c29 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:11.105668 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85697c03-6821-407a-beb7-6b74f99afeab None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:11.173997 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:11.175276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.175749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.176251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.178334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:11.181062 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:11.185159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 109/213] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:11.190795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:11.276486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7d87a-e03a-40d1-8056-0111bf464c29 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:11.326275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ff731-b635-4879-a5df-666d7afb23b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:11.327196 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff731-b635-4879-a5df-666d7afb23b7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.327520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff731-b635-4879-a5df-666d7afb23b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.327788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff731-b635-4879-a5df-666d7afb23b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.340765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccb564-6d6a-47d8-954d-741133cd6016 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:11.341015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ff731-b635-4879-a5df-666d7afb23b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-f01ff731-b635-4879-a5df-666d7afb23b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:11.346234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:11.347952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:11.349589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:11.352322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 105/214] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:11.353850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeeff8-ae86-4369-9b3d-1fe7df9986a3 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.354681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeeff8-ae86-4369-9b3d-1fe7df9986a3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.354681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeeff8-ae86-4369-9b3d-1fe7df9986a3 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.356382 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 106/215] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:11.356598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aeeff8-ae86-4369-9b3d-1fe7df9986a3 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:11.386466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45c76d-e0dd-48f5-9b6d-8eb514794630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:11.387085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45c76d-e0dd-48f5-9b6d-8eb514794630 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.387363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45c76d-e0dd-48f5-9b6d-8eb514794630 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.387630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45c76d-e0dd-48f5-9b6d-8eb514794630 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.388282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45c76d-e0dd-48f5-9b6d-8eb514794630 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:11.389780 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45c76d-e0dd-48f5-9b6d-8eb514794630 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:11.457430 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:11.505278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 110/216] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:11 2022] POST /identity/v3/users => generated 297 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:11.508451 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291898d-6077-40f8-8c77-9e2e16834d78 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.509240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291898d-6077-40f8-8c77-9e2e16834d78 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.510045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291898d-6077-40f8-8c77-9e2e16834d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.521029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aeeff8-ae86-4369-9b3d-1fe7df9986a3 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:11.549063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e291898d-6077-40f8-8c77-9e2e16834d78 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:11.551964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e291898d-6077-40f8-8c77-9e2e16834d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:11.565843 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:11.566723 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.567100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.567471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.568262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:11.568666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:11.570521 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:11.580871 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:11.581546 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 107/217] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:11.587273 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:11.719977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 111/218] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:11.751288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ce4aa-b1d5-47dc-ba17-69c466894128 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:11.776926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:11.812398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353f078-d03b-4b01-a0ba-7236cb40c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:11.814631 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f078-d03b-4b01-a0ba-7236cb40c89a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.815435 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f078-d03b-4b01-a0ba-7236cb40c89a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.816215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353f078-d03b-4b01-a0ba-7236cb40c89a None admin] PATH_INFO: `/v3/users/6f9f216ceea242c38043e8959d13d76c` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.848580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353f078-d03b-4b01-a0ba-7236cb40c89a None admin] RBAC: Authorizing `identity:get_user(user_id=6f9f216ceea242c38043e8959d13d76c)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:11.851831 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353f078-d03b-4b01-a0ba-7236cb40c89a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:11.855378 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 108/219] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:11 2022] GET /identity/v3/users/6f9f216ceea242c38043e8959d13d76c => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:11.862606 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291898d-6077-40f8-8c77-9e2e16834d78 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:11.980756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da61ba-9a51-4c2b-a9a1-914d7acabcb5 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.982589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da61ba-9a51-4c2b-a9a1-914d7acabcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.983357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da61ba-9a51-4c2b-a9a1-914d7acabcb5 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:11.986780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 109/220] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:11.996321 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9056c-fab2-4ee5-b5fe-63541046c3d1 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:11.997245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9056c-fab2-4ee5-b5fe-63541046c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:11.998758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9056c-fab2-4ee5-b5fe-63541046c3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:12.031849 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291898d-6077-40f8-8c77-9e2e16834d78 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:12.037730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-0ed9056c-fab2-4ee5-b5fe-63541046c3d1 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:12.038930 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0ed9056c-fab2-4ee5-b5fe-63541046c3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:12.091873 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:12.092509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:12.092790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:12.093065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:12.114776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:12.124329 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 110/221] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:12.129218 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9056c-fab2-4ee5-b5fe-63541046c3d1 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:12.175740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:12.177590 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:12.306111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 112/222] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:57:12.318506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:12.374344 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9056c-fab2-4ee5-b5fe-63541046c3d1 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:12.377955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:12.428950 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:12.461791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:12.462579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:12.462733 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:12.463016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:12.483641 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:12.489282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:12.490381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:12.491149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:12.498653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 111/223] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:57:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:12.511769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:12.520023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2512ea9-b16e-4ab1-9ac2-1b51bc48dbfd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:12.582383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:12.583518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:12.583834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:12.584064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:12.585473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:12.586981 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:12.588701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 113/224] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:12 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:12.598675 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:12.717993 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5d9ae-5a5c-4f03-a80f-f729db91102c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:12.736796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:12.756256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:12.756887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:12.757168 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:12.757443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:12.758222 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=6f9f216ceea242c38043e8959d13d76c) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:12.758537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:12.760006 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:12.766721 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 114/225] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:12 2022] GET /identity/v3/role_assignments?user.id=6f9f216ceea242c38043e8959d13d76c&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:12.767629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:12.845673 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739d69b-6d09-408e-8128-c563066e0883 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:12.890562 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f684d2a-c5d8-4e89-bb68-be7b51c2066f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:12.934218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:12.934909 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:12.935309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:12.935686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:12.943122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:12.943763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:12.944050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:12.944324 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:12.945295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:12.945632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:12.947094 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:12.950585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:12.956278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:12.958100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:12.959417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 115/226] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:12.968463 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:12.975478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 112/227] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:12.987747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:13.044994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:13.061120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:13.160720 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986119d-5c00-4631-9b4d-fe124066913a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:13.208071 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:13.208816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:13.209111 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:13.209393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] PATH_INFO: `/v3/users/a6f6e9287c18489c84cbb267d909635b` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:13.214582 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] RBAC: Authorizing `identity:get_user(user_id=a6f6e9287c18489c84cbb267d909635b)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:13.216228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:13.218141 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b9f1f-ab2c-47e4-a89b-0a377358cfe1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:13.218297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 113/228] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:12 2022] GET /identity/v3/users/a6f6e9287c18489c84cbb267d909635b => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:13.224517 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:13.266797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:13.267789 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:13.268290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:13.268765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:13.269684 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:13.270217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:13.271862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:13.288830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 116/229] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:13.297358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:13.346005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:13.354421 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4c400-8eef-4f23-827c-d8322a6dd85a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:13.458539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:13.459475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:13.459905 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:13.460017 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:13.465638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720737-2760-4529-8407-9bd402f31e89 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:13.466282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:13.474519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:13.476253 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:13.478225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 114/230] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:13 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:13.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:13.513709 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:13.514384 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:13.514669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:13.514946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] PATH_INFO: `/v3/users/a6f6e9287c18489c84cbb267d909635b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:13.518757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] RBAC: Authorizing `identity:get_user(user_id=a6f6e9287c18489c84cbb267d909635b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:13.520481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:13.527101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 117/231] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:13 2022] GET /identity/v3/users/a6f6e9287c18489c84cbb267d909635b => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:13.532799 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:13.705712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42d17-dd7a-4226-a163-d6ef0a4f753c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:13.753308 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:13.753999 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:13.754286 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:13.754566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:13.759164 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:13.760661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:13.764503 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f02642-8ffe-4f63-a568-f1c586c32f4d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:13.768316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 118/232] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:13 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:13.772762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:13.814255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:13.899240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16ccdb-1c0c-4e4d-9aea-ead8e7872f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:13.899933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16ccdb-1c0c-4e4d-9aea-ead8e7872f76 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:13.900218 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16ccdb-1c0c-4e4d-9aea-ead8e7872f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:13.900501 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16ccdb-1c0c-4e4d-9aea-ead8e7872f76 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/a6f6e9287c18489c84cbb267d909635b/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:13.905695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16ccdb-1c0c-4e4d-9aea-ead8e7872f76 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b1e5e0ec4174aac9379046a27e7794a, user_id=a6f6e9287c18489c84cbb267d909635b, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:13.907782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16ccdb-1c0c-4e4d-9aea-ead8e7872f76 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:13.934637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374978b7-3130-435b-9e6e-d86aee060948 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:13.973945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4ed11-fd15-4322-b729-f9860d1537dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:13.974540 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4ed11-fd15-4322-b729-f9860d1537dd None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:13.974824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4ed11-fd15-4322-b729-f9860d1537dd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:13.975104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4ed11-fd15-4322-b729-f9860d1537dd None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/a6f6e9287c18489c84cbb267d909635b/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:13.977346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4ed11-fd15-4322-b729-f9860d1537dd None admin] RBAC: Authorizing `identity:create_grant(project_id=8b1e5e0ec4174aac9379046a27e7794a, user_id=a6f6e9287c18489c84cbb267d909635b, role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:13.978909 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4ed11-fd15-4322-b729-f9860d1537dd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:14.008692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 115/233] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:57:13 2022] PUT /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/a6f6e9287c18489c84cbb267d909635b/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:57:14.121265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 119/234] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:57:13 2022] PUT /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/a6f6e9287c18489c84cbb267d909635b/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:14.180322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112fe6-4c5e-4464-a347-f1126a77be40 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:14.180691 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112fe6-4c5e-4464-a347-f1126a77be40 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:14.180893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112fe6-4c5e-4464-a347-f1126a77be40 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:14.182485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 116/235] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:14.193071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdbbb5-e698-4abb-b6c9-12acfea8815e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:14.193396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdbbb5-e698-4abb-b6c9-12acfea8815e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:14.193659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdbbb5-e698-4abb-b6c9-12acfea8815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:14.218343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4dbdbbb5-e698-4abb-b6c9-12acfea8815e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:14.220266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4dbdbbb5-e698-4abb-b6c9-12acfea8815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:14.309080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 120/236] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:14.483364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228d972-11a6-40fc-a55d-3d37b4bd8ceb None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:14.483907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228d972-11a6-40fc-a55d-3d37b4bd8ceb None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:14.483907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228d972-11a6-40fc-a55d-3d37b4bd8ceb None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:14.486281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 117/237] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:14.494173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5288f-8d14-47e5-aaeb-d12fc75c2461 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:14.494470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5288f-8d14-47e5-aaeb-d12fc75c2461 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:14.494743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5288f-8d14-47e5-aaeb-d12fc75c2461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:14.518586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-59a5288f-8d14-47e5-aaeb-d12fc75c2461 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:14.519902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-59a5288f-8d14-47e5-aaeb-d12fc75c2461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:14.554837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:14.631361 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 121/238] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:14.742639 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228d972-11a6-40fc-a55d-3d37b4bd8ceb None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:14.771095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:14.893043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228d972-11a6-40fc-a55d-3d37b4bd8ceb None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:14.943264 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:14.943919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:14.944205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:14.944484 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:14.950466 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:14.951952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:14.954630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 118/239] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:14.962700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5288f-8d14-47e5-aaeb-d12fc75c2461 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:14.977331 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:15.080934 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:15.095071 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6790e2-d1ef-4bd4-9921-29c8242b24a9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:15.147159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:15.147732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:15.148015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:15.148291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:15.170146 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:15.179402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:15.180374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:15.181166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:15.185435 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 119/240] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:15.191488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:15.243027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5288f-8d14-47e5-aaeb-d12fc75c2461 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:15.315255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:15.316289 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:15.316795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:15.317267 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:15.318184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:15.329633 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:15.353864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26389-2ea7-47ca-88b7-e786460cc712 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:15.388694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:15.389414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:15.389799 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:15.390072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:15.391318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:15.391628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:15.393032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:15.403416 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 120/241] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:15.409858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:15.423752 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:15.480327 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 122/242] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:14 2022] POST /identity/v3/users => generated 293 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:15.536462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618bca1-a9cc-47fc-9f32-b9bcfb2f6c92 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:15.576851 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccad1f3-ab0c-42ba-aa11-cc68713e242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:15.577517 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccad1f3-ab0c-42ba-aa11-cc68713e242f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:15.577832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccad1f3-ab0c-42ba-aa11-cc68713e242f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:15.578138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccad1f3-ab0c-42ba-aa11-cc68713e242f None admin] PATH_INFO: `/v3/users/24d0197fe7f14384b6b829c27e6e17f6` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:15.595136 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccad1f3-ab0c-42ba-aa11-cc68713e242f None admin] RBAC: Authorizing `identity:get_user(user_id=24d0197fe7f14384b6b829c27e6e17f6)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:15.596710 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccad1f3-ab0c-42ba-aa11-cc68713e242f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:15.598485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 121/243] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:15 2022] GET /identity/v3/users/24d0197fe7f14384b6b829c27e6e17f6 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:15.604616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:15.649566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:15.836673 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a5d41-18c9-4d8c-b574-8cf8c04a74ac None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:15.836932 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a5d41-18c9-4d8c-b574-8cf8c04a74ac None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:15.837214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a5d41-18c9-4d8c-b574-8cf8c04a74ac None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:15.839029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 122/244] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:15.848052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c6a94-5414-4875-9fa1-a632b46a6f25 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:15.848349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c6a94-5414-4875-9fa1-a632b46a6f25 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:15.848601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c6a94-5414-4875-9fa1-a632b46a6f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:15.873848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8a9f0-d85b-4489-b929-a1861c892e56 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:15.875415 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-3c8c6a94-5414-4875-9fa1-a632b46a6f25 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:15.876648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3c8c6a94-5414-4875-9fa1-a632b46a6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:15.948330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:15.950743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:15.951754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:15.953068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:15.956313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:15.969783 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 123/245] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:16.094662 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:16.096083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:16.116447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:16.221452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 123/246] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:57:16.226880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c6a94-5414-4875-9fa1-a632b46a6f25 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:16.349796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c6a94-5414-4875-9fa1-a632b46a6f25 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:16.390156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7456c9-b619-4d3a-94d8-52f7684d6d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:16.391266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7456c9-b619-4d3a-94d8-52f7684d6d74 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:16.391756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7456c9-b619-4d3a-94d8-52f7684d6d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:16.392260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7456c9-b619-4d3a-94d8-52f7684d6d74 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:16.396026 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7456c9-b619-4d3a-94d8-52f7684d6d74 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:16.397847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7456c9-b619-4d3a-94d8-52f7684d6d74 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:16.400186 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 124/247] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:16 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:16.406017 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:16.490042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f6089-b9b6-41f9-ad8b-d718e6bb42d2 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:16.490531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f6089-b9b6-41f9-ad8b-d718e6bb42d2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:16.491012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f6089-b9b6-41f9-ad8b-d718e6bb42d2 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:16.498063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 125/248] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:16.500908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a32c1b-c0f0-449d-aef9-85b16ef2b83f None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:16.501401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a32c1b-c0f0-449d-aef9-85b16ef2b83f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:16.506154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a32c1b-c0f0-449d-aef9-85b16ef2b83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:16.526506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cabf16-e3a1-4f50-ad0b-35da9690d13f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:16.543240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:16.543429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-84a32c1b-c0f0-449d-aef9-85b16ef2b83f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:16.546689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-84a32c1b-c0f0-449d-aef9-85b16ef2b83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:16.581768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:16.582946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:16.583435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:16.583929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:16.585370 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] RBAC: Adding query filter params (role.id=79a475679b454999b9d77c6405a7f0a7, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=24d0197fe7f14384b6b829c27e6e17f6) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:16.586907 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:16.588720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:16.601512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 124/249] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:16 2022] GET /identity/v3/role_assignments?user.id=24d0197fe7f14384b6b829c27e6e17f6&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=79a475679b454999b9d77c6405a7f0a7 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:16.602883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:16.646239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:16.674115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 126/250] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:16.675423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d449b-5f8d-46fd-8e03-f2c1f8475b0d None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:16.675712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d449b-5f8d-46fd-8e03-f2c1f8475b0d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:16.675996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d449b-5f8d-46fd-8e03-f2c1f8475b0d None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:16.681741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 127/251] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:16.687599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1b46-caac-4d97-9d34-9e23d41a9a58 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:16.688384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1b46-caac-4d97-9d34-9e23d41a9a58 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:16.688681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1b46-caac-4d97-9d34-9e23d41a9a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:16.734035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-a41d1b46-caac-4d97-9d34-9e23d41a9a58 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:16.735832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-a41d1b46-caac-4d97-9d34-9e23d41a9a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:16.818157 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d80e3-9155-4e62-b476-57eee192f584 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:16.897530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:16.898655 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:16.899151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:16.899640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:16.936742 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:16.951423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 128/252] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:16.954786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:16.955672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:16.956454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:16.957049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 125/253] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:16.966830 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d1b46-caac-4d97-9d34-9e23d41a9a58 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:17.161512 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d1b46-caac-4d97-9d34-9e23d41a9a58 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:17.210262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:17.224520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:17.225598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:17.226123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:17.226612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:17.227844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:17.228374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:17.231138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:17.248617 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 129/254] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:17.259098 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:17.332631 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:17.379913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:17.477490 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a6e43-ac84-4512-9e17-18acc36e45bc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:17.606844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:17.607924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:17.608423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:17.608912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] PATH_INFO: `/v3/users/6f9f216ceea242c38043e8959d13d76c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:17.621734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] RBAC: Authorizing `identity:get_user(user_id=6f9f216ceea242c38043e8959d13d76c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:17.624527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:17.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 126/255] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:17 2022] GET /identity/v3/users/6f9f216ceea242c38043e8959d13d76c => generated 297 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:17.629644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:17.714504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f0dbb-9175-47ec-844d-dc749e713e3e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:17.828876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:17.830020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:17.830523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:17.831015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:17.877142 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:17.893645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:17.894651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:17.895529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:17.898807 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 130/256] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 520 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:17.901996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:17.947885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a3fb5-f53f-4003-8722-b59fc0d673d7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:17.951247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:18.004139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:18.037937 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:18.038607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:18.038891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:18.039176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:18.068581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:18.074372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:18.075297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:18.076101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 127/257] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:18.076838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:18.135065 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bcd54-499a-4e2d-b286-efcdfe93c102 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:18.206975 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625840d-088b-497b-930b-da58227e074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:18.208089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625840d-088b-497b-930b-da58227e074c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:18.208580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625840d-088b-497b-930b-da58227e074c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:18.208926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625840d-088b-497b-930b-da58227e074c None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:18.210854 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625840d-088b-497b-930b-da58227e074c None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:18.213333 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625840d-088b-497b-930b-da58227e074c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:18.216521 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 131/258] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:17 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:18.218493 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625840d-088b-497b-930b-da58227e074c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:18.251902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac244d-15a5-4b79-ba94-d522e9b155eb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:18.334919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:18.335940 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:18.336432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:18.336915 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:18.337840 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:18.338490 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:18.340242 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:18.362041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 128/259] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:18.363224 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:18.420660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625840d-088b-497b-930b-da58227e074c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:18.425879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:18.459481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:18.490847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:18.491786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:18.492652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:18.493167 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:18.494878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:18.495453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:18.498118 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:18.513946 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 132/260] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:18.515597 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:18.607226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8a0bd-ea21-4ab9-9c1f-e314f0ef7d6f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:18.657346 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec471f77-ac3d-4876-9f39-2341d5b07ddd None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:18.685177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea0177-eb4b-49d4-8790-6f129470a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:18.685960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea0177-eb4b-49d4-8790-6f129470a2c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:18.686654 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea0177-eb4b-49d4-8790-6f129470a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:18.687141 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea0177-eb4b-49d4-8790-6f129470a2c7 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/6f9f216ceea242c38043e8959d13d76c/roles/bddb19f21b7f472589b9154cb2aeeecc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:18.690139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea0177-eb4b-49d4-8790-6f129470a2c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b1e5e0ec4174aac9379046a27e7794a, user_id=6f9f216ceea242c38043e8959d13d76c, role_id=bddb19f21b7f472589b9154cb2aeeecc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:18.691999 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cea0177-eb4b-49d4-8790-6f129470a2c7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:18.719985 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:18.721063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:18.721532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:18.722051 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] PATH_INFO: `/v3/users/a6f6e9287c18489c84cbb267d909635b` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:18.724102 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] RBAC: Authorizing `identity:get_user(user_id=a6f6e9287c18489c84cbb267d909635b)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:18.726960 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:18.730066 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 133/261] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:18 2022] GET /identity/v3/users/a6f6e9287c18489c84cbb267d909635b => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:18.733669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:18.776204 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 129/262] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:57:18 2022] PUT /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/6f9f216ceea242c38043e8959d13d76c/roles/bddb19f21b7f472589b9154cb2aeeecc => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:18.783914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349d9b-1ec6-49c2-be1c-38aef842008f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:18.783914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349d9b-1ec6-49c2-be1c-38aef842008f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:18.783914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349d9b-1ec6-49c2-be1c-38aef842008f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:18.783914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 130/263] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:18.783914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349d9b-1ec6-49c2-be1c-38aef842008f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:18.854869 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:18.956029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e143-5b36-4433-b6c8-62071c8ce5db None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:19.002634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:19.003436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251df2-19f8-4ccd-a054-09670cba2954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:19.003904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251df2-19f8-4ccd-a054-09670cba2954 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.004240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251df2-19f8-4ccd-a054-09670cba2954 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.004546 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251df2-19f8-4ccd-a054-09670cba2954 None admin] PATH_INFO: `/v3/users/a6f6e9287c18489c84cbb267d909635b` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.006141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6251df2-19f8-4ccd-a054-09670cba2954 None admin] RBAC: Authorizing `identity:get_user(user_id=a6f6e9287c18489c84cbb267d909635b)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:19.007688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6251df2-19f8-4ccd-a054-09670cba2954 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:19.009429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 134/264] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:18 2022] GET /identity/v3/users/a6f6e9287c18489c84cbb267d909635b => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:19.011595 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8935698-c238-4206-95e5-dc027bea50db None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.011595 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8935698-c238-4206-95e5-dc027bea50db None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.011872 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8935698-c238-4206-95e5-dc027bea50db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.037874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-b8935698-c238-4206-95e5-dc027bea50db None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:19.039373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-b8935698-c238-4206-95e5-dc027bea50db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:19.049500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8349d9b-1ec6-49c2-be1c-38aef842008f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:19.144169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 135/265] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:19.144838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8935698-c238-4206-95e5-dc027bea50db None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:19.146989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a223db-5d4f-4966-a95e-7df238b1da65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:19.148442 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a223db-5d4f-4966-a95e-7df238b1da65 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.150395 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a223db-5d4f-4966-a95e-7df238b1da65 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.151263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a223db-5d4f-4966-a95e-7df238b1da65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.213787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:19.296007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8935698-c238-4206-95e5-dc027bea50db None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:19.330610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a223db-5d4f-4966-a95e-7df238b1da65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:19.334079 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a223db-5d4f-4966-a95e-7df238b1da65 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:19.353592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43513cec-970f-45cb-9066-81d1879bdaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:19.354252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43513cec-970f-45cb-9066-81d1879bdaff None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.354548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43513cec-970f-45cb-9066-81d1879bdaff None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.354809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43513cec-970f-45cb-9066-81d1879bdaff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.467035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:19.486332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 131/266] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:19.487741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1d5da-87bc-4268-bd3b-b5f0ce9efb5a None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.488060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1d5da-87bc-4268-bd3b-b5f0ce9efb5a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.488288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1d5da-87bc-4268-bd3b-b5f0ce9efb5a None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.489593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 132/267] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:19.490743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7107c9-76d5-4d1a-bb1d-4e05eb542606 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.491066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7107c9-76d5-4d1a-bb1d-4e05eb542606 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.491339 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7107c9-76d5-4d1a-bb1d-4e05eb542606 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.492645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 133/268] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:19.499600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7107c9-76d5-4d1a-bb1d-4e05eb542606 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:19.553920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43513cec-970f-45cb-9066-81d1879bdaff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:19.556730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43513cec-970f-45cb-9066-81d1879bdaff None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:19.635336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:19.644931 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7107c9-76d5-4d1a-bb1d-4e05eb542606 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:19.695298 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbda20e-d6a1-428c-897e-3ea6fb8bbb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:19.696801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbda20e-d6a1-428c-897e-3ea6fb8bbb4a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.697246 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbda20e-d6a1-428c-897e-3ea6fb8bbb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.698088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbda20e-d6a1-428c-897e-3ea6fb8bbb4a None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.699576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbda20e-d6a1-428c-897e-3ea6fb8bbb4a None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:19.701520 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbda20e-d6a1-428c-897e-3ea6fb8bbb4a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:19.704287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 134/269] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:19 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:19.705807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a33f13-321b-4397-bef4-d7e0287ce584 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.706265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a33f13-321b-4397-bef4-d7e0287ce584 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.706572 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a33f13-321b-4397-bef4-d7e0287ce584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.726716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-11a33f13-321b-4397-bef4-d7e0287ce584 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:19.727826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-11a33f13-321b-4397-bef4-d7e0287ce584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:19.753543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 136/270] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:19.753543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b7d37-f67b-4231-b55c-0a175f70208f None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:19.753543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b7d37-f67b-4231-b55c-0a175f70208f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:19.753543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b7d37-f67b-4231-b55c-0a175f70208f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:19.784409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-d47b7d37-f67b-4231-b55c-0a175f70208f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:19.787483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d47b7d37-f67b-4231-b55c-0a175f70208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:19.806535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:19.849635 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 135/271] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:19.849635 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a33f13-321b-4397-bef4-d7e0287ce584 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:19.931995 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 137/272] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:19.933724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b7d37-f67b-4231-b55c-0a175f70208f None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:20.131640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a33f13-321b-4397-bef4-d7e0287ce584 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:20.138135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:20.149294 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b7d37-f67b-4231-b55c-0a175f70208f None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:20.175224 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:20.176429 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:20.176950 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:20.177411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:20.178786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:20.180257 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:20.181920 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 136/273] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:20.182689 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:20.253335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:20.254990 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:20.255493 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:20.255978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:20.257337 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=a6f6e9287c18489c84cbb267d909635b) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:20.257917 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:20.260064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:20.263583 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:20.278355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 138/274] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:19 2022] GET /identity/v3/role_assignments?user.id=a6f6e9287c18489c84cbb267d909635b&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:20.280869 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:20.324232 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee834a-e155-4cac-8204-38e30ebab4c6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:20.401832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:20.402873 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:20.403340 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:20.403796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:20.405368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:20.407251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:20.409324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 137/275] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:20 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:20.410401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:20.572315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3759913-cb4a-458f-9963-c8f884888fb2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:20.618949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:20.619801 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:20.620089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:20.620368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:20.622038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6513d7-9336-46db-8699-10bbdc5e90fd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:20.635069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:20.641574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:20.643837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 139/276] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:20.645398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:20.651238 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:20.672704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:20.705311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:20.706688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:20.707184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:20.707664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:20.710127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:20.714232 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:20.819070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c3b07-009b-4b68-9f48-43d54c9652a1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:20.863884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:20.864613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:20.864951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:20.865297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:20.866161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] RBAC: Adding query filter params (role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=a6f6e9287c18489c84cbb267d909635b) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:20.866557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:20.868086 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:20.875162 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 140/277] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:20 2022] GET /identity/v3/role_assignments?user.id=a6f6e9287c18489c84cbb267d909635b&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:20.876229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:20.958737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:21.005315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c93327-cca4-4dff-92b2-d339edad939b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:21.044030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:21.044696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.044976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.045255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.058215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:21.064533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:21.065535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:21.066418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 141/278] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:21.071371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:21.127014 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 138/279] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:20 2022] POST /identity/v3/users => generated 293 bytes in 717 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:21.132855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:21.208649 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840adc0-b199-4cc2-b7a6-e8feb97889db None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:21.242468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:21.258392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:21.259618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.259618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.259807 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.260628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:21.260941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:21.263112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:21.280782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 142/280] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:21.285977 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:21.310999 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c020d-35d7-4234-ae88-393d79aae2b8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:21.402593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:21.403858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.404188 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.404658 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.406022 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:21.407283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:21.410081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:21.412392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47e350-04ab-4dfa-b250-a07520f747ba None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:21.427685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 139/281] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:21.434355 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:21.471941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e273116-e4fc-44f7-adb2-99b108cf2bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:21.473057 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273116-e4fc-44f7-adb2-99b108cf2bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.474553 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273116-e4fc-44f7-adb2-99b108cf2bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.475222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273116-e4fc-44f7-adb2-99b108cf2bb9 None admin] PATH_INFO: `/v3/users/8846c7627dd04387be52adc1c67b37ef` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.501132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:21.503109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e273116-e4fc-44f7-adb2-99b108cf2bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=8846c7627dd04387be52adc1c67b37ef)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:21.505471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e273116-e4fc-44f7-adb2-99b108cf2bb9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:21.507562 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 143/282] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:21 2022] GET /identity/v3/users/8846c7627dd04387be52adc1c67b37ef => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:21.508735 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1e56-0354-439f-b393-c369539f82f0 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.509021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1e56-0354-439f-b393-c369539f82f0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.509297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1e56-0354-439f-b393-c369539f82f0 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.510669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 144/283] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:21.517143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd0da1-8f06-4955-8e1a-7a3ad6a76d32 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.517528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd0da1-8f06-4955-8e1a-7a3ad6a76d32 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.517731 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd0da1-8f06-4955-8e1a-7a3ad6a76d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.538195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-27dd0da1-8f06-4955-8e1a-7a3ad6a76d32 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:21.539530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-27dd0da1-8f06-4955-8e1a-7a3ad6a76d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:21.597921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:21.606727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df9291-a4e4-40a3-b1d0-206f9ea2c7da None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:21.660449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:21.661373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.661841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.662279 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] PATH_INFO: `/v3/users/24d0197fe7f14384b6b829c27e6e17f6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.663652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] RBAC: Authorizing `identity:get_user(user_id=24d0197fe7f14384b6b829c27e6e17f6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:21.665184 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 145/284] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:21.665503 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:21.666879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd0da1-8f06-4955-8e1a-7a3ad6a76d32 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:21.667844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 140/285] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:21 2022] GET /identity/v3/users/24d0197fe7f14384b6b829c27e6e17f6 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:21.676605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:21.800556 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b674f-abfa-43fd-9631-85bb04533492 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:21.814181 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:21.842814 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:21.844340 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.844767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.845210 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.846908 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:21.848579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:21.850600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 141/286] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:21 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:21.863071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:21.880777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd0da1-8f06-4955-8e1a-7a3ad6a76d32 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:21.957393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:21.958105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:21.958384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:21.958655 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:21.959453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, user.id=8846c7627dd04387be52adc1c67b37ef, scope.system=all) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:21.959764 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:21.961234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:21.975989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 146/287] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Sep 1 11:57:21 2022] GET /identity/v3/role_assignments?user.id=8846c7627dd04387be52adc1c67b37ef&scope.system=all&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:22.048086 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248faa7b-72da-42ae-8442-9f60123de375 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:22.109423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:22.110067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:22.110347 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:22.110624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/24d0197fe7f14384b6b829c27e6e17f6/roles/79a475679b454999b9d77c6405a7f0a7` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:22.118667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b1e5e0ec4174aac9379046a27e7794a, user_id=24d0197fe7f14384b6b829c27e6e17f6, role_id=79a475679b454999b9d77c6405a7f0a7)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:22.120162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:22.202802 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 142/288] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:57:21 2022] PUT /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/users/24d0197fe7f14384b6b829c27e6e17f6/roles/79a475679b454999b9d77c6405a7f0a7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:57:22.242220 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:22.325698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:22.499167 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a150b-fdda-4e0d-b39d-16fda89e8d66 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:22.580962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:22.582037 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:22.582528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:22.583022 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:22.611638 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:22.621377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:22.622340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:22.623154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:22.629755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 147/289] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:22.634768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:22.692748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:22.769326 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f71013-cdb2-4d3d-b4c8-d9d0c19ca024 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:22.811163 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:22.811793 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:22.812071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:22.812364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:22.813328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:22.813652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:22.815055 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:22.825473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 143/290] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:22.831907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:22.988111 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:22.996302 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ffa56-eddf-4263-93b4-7f417907197d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:23.048577 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:23.049526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:23.050023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:23.050485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] PATH_INFO: `/v3/users/6f9f216ceea242c38043e8959d13d76c` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:23.051854 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] RBAC: Authorizing `identity:get_user(user_id=6f9f216ceea242c38043e8959d13d76c)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:23.053510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:23.055446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 148/291] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:22 2022] GET /identity/v3/users/6f9f216ceea242c38043e8959d13d76c => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:23.061470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:23.222881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:23.229106 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4fb6d-7ad3-426e-9265-9d94c8ee1fad None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:23.287804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:23.288445 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:23.288772 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:23.289060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:23.389368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:23.391334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:23.542211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:23.548335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 144/292] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:23.555497 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:23.724762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908faa2e-84b9-42cc-8c39-e3c33a3bb312 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:23.780394 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26adf03-6761-431b-9861-5e5d30c2ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:23.781675 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26adf03-6761-431b-9861-5e5d30c2ed10 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:23.781987 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26adf03-6761-431b-9861-5e5d30c2ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:23.782254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26adf03-6761-431b-9861-5e5d30c2ed10 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:23.783459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26adf03-6761-431b-9861-5e5d30c2ed10 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:23.784940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26adf03-6761-431b-9861-5e5d30c2ed10 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:23.786967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 149/293] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:23 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:23.792663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:23.946628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2edb6a-c346-4615-b28a-466fcacee486 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:23.995162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb0289-a32a-46a8-9d82-6b3cbb479b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:23.995786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb0289-a32a-46a8-9d82-6b3cbb479b04 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:23.996070 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb0289-a32a-46a8-9d82-6b3cbb479b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:23.996347 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb0289-a32a-46a8-9d82-6b3cbb479b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:23.997093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb0289-a32a-46a8-9d82-6b3cbb479b04 None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=6f9f216ceea242c38043e8959d13d76c) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:23.997405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb0289-a32a-46a8-9d82-6b3cbb479b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:23.998926 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb0289-a32a-46a8-9d82-6b3cbb479b04 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:24.006397 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 145/294] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:23 2022] GET /identity/v3/role_assignments?user.id=6f9f216ceea242c38043e8959d13d76c&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:24.116731 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b52c9-ed75-4633-b363-4c07367868f2 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:24.117297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b52c9-ed75-4633-b363-4c07367868f2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:24.117297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b52c9-ed75-4633-b363-4c07367868f2 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:24.119096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 150/295] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:24.126911 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b67b4-c84f-4097-9656-16cabc92def2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:24.127218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b67b4-c84f-4097-9656-16cabc92def2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:24.127467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b67b4-c84f-4097-9656-16cabc92def2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:24.153643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-070b67b4-c84f-4097-9656-16cabc92def2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:24.154937 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-070b67b4-c84f-4097-9656-16cabc92def2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:24.240521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:24.256224 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 146/296] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:24.464405 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a08e7-91ca-4a64-a584-77fb26c69876 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:24.464757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a08e7-91ca-4a64-a584-77fb26c69876 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:24.464962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a08e7-91ca-4a64-a584-77fb26c69876 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:24.466748 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 151/297] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:24.473526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17c722-7535-477a-9712-791797237633 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:24.473895 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17c722-7535-477a-9712-791797237633 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:24.474128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17c722-7535-477a-9712-791797237633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:24.502637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8c17c722-7535-477a-9712-791797237633 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:24.504111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8c17c722-7535-477a-9712-791797237633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:24.553950 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a08e7-91ca-4a64-a584-77fb26c69876 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:24.613991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 147/298] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:24.646819 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:24.783758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a08e7-91ca-4a64-a584-77fb26c69876 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:24.826783 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e2fdd-e416-4a41-866f-629bcfeb3945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:24.827327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2fdd-e416-4a41-866f-629bcfeb3945 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:24.827596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2fdd-e416-4a41-866f-629bcfeb3945 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:24.827869 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2fdd-e416-4a41-866f-629bcfeb3945 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:24.831202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e2fdd-e416-4a41-866f-629bcfeb3945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:24.831439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e2fdd-e416-4a41-866f-629bcfeb3945 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:24.833375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 152/299] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:24.838865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17c722-7535-477a-9712-791797237633 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:24.926764 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521613c-8412-4d85-b896-ad349443efa5 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:24.927673 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521613c-8412-4d85-b896-ad349443efa5 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:24.927951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521613c-8412-4d85-b896-ad349443efa5 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:24.930092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 153/300] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:24.939640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9a40-754e-4695-a0da-5bed9135705f None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:24.940338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9a40-754e-4695-a0da-5bed9135705f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:24.940720 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9a40-754e-4695-a0da-5bed9135705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:24.960564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17c722-7535-477a-9712-791797237633 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:24.967949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-80bd9a40-754e-4695-a0da-5bed9135705f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:24.970837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-80bd9a40-754e-4695-a0da-5bed9135705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:24.986183 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:25.008579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:25.009365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.010341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.010733 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] PATH_INFO: `/v3/groups` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.011527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:25.013515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:25.084259 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 154/301] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:25.085586 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd9a40-754e-4695-a0da-5bed9135705f None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:25.098258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 148/302] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Sep 1 11:57:24 2022] POST /identity/v3/groups => generated 229 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:25.153784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:25.291622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd9a40-754e-4695-a0da-5bed9135705f None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:25.341739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:25.341739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.341739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.341739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.352891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:25.357956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:25.360114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:25.362025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:25.363208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:25.363860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 155/303] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:25.368721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:25.490717 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c1d58-02dd-4f33-9bd6-2a3accfa53f1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:25.502893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:25.518277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cae86-7598-4845-934f-01e1a34d7eb5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:25.543375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:25.544740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.545218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.545953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.560348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01d230-f1db-4b1e-8904-58e343ffa1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:25.560953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01d230-f1db-4b1e-8904-58e343ffa1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.561247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01d230-f1db-4b1e-8904-58e343ffa1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.561543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01d230-f1db-4b1e-8904-58e343ffa1d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.562230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01d230-f1db-4b1e-8904-58e343ffa1d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:25.562560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01d230-f1db-4b1e-8904-58e343ffa1d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:25.563944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01d230-f1db-4b1e-8904-58e343ffa1d9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:25.568605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:25.573537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 156/304] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:25.573537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dd179-d62b-4f4a-83c5-ff3b2f870a5e None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.575300 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dd179-d62b-4f4a-83c5-ff3b2f870a5e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.575380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dd179-d62b-4f4a-83c5-ff3b2f870a5e None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.576693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 157/305] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:25.582410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f8f3f-3d6e-41d5-bfd1-a87eaaf37f3b None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.582727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f8f3f-3d6e-41d5-bfd1-a87eaaf37f3b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.583005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f8f3f-3d6e-41d5-bfd1-a87eaaf37f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:25.589692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:25.590629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:25.591440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:25.597953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 149/306] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:25.597953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:25.603342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-e64f8f3f-3d6e-41d5-bfd1-a87eaaf37f3b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:25.604556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-e64f8f3f-3d6e-41d5-bfd1-a87eaaf37f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:25.676685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:25.684592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 158/307] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:25.685470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f8f3f-3d6e-41d5-bfd1-a87eaaf37f3b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:25.787806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c161e6b-fe1e-4dc8-b21b-f3f2f3c0b115 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:25.865203 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:25.899084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:25.899547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f8f3f-3d6e-41d5-bfd1-a87eaaf37f3b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:25.900831 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.901540 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.902278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] PATH_INFO: `/v3/users/7db38f74a07b4e2eb095fed2c95e5fe2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.927179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] RBAC: Authorizing `identity:get_user(user_id=7db38f74a07b4e2eb095fed2c95e5fe2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:25.930285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:25.938236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 150/308] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:25 2022] GET /identity/v3/users/7db38f74a07b4e2eb095fed2c95e5fe2 => generated 293 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:25.944393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:25.964396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:25.965363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:25.965860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:25.966325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:25.967489 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:25.968255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:25.970619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:25.987007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 159/309] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:25.996373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:26.112924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:26.117894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021825d-8671-4878-a167-1d4d4298e7e2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:26.152564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6baf30-8667-42de-94b7-d7fe7e2c94f6 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:26.160454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:26.161432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:26.161920 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:26.162372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:26.163497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] RBAC: Adding query filter params (role.id=79a475679b454999b9d77c6405a7f0a7, user.id=7db38f74a07b4e2eb095fed2c95e5fe2, scope.system=all) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:26.164019 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:26.165820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:26.181638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 151/310] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Sep 1 11:57:25 2022] GET /identity/v3/role_assignments?user.id=7db38f74a07b4e2eb095fed2c95e5fe2&scope.system=all&role.id=79a475679b454999b9d77c6405a7f0a7 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:26.181638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:26.207669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:26.208950 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:26.209475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:26.210089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] PATH_INFO: `/v3/users/8846c7627dd04387be52adc1c67b37ef` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:26.212509 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] RBAC: Authorizing `identity:get_user(user_id=8846c7627dd04387be52adc1c67b37ef)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:26.215843 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:26.226354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 160/311] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:25 2022] GET /identity/v3/users/8846c7627dd04387be52adc1c67b37ef => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:26.230666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:26.265011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:26.307518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdddc-ac3c-4c1b-9c09-acebd2d9e160 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:26.349648 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:26.350286 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:26.350562 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:26.350832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:26.364528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:26.369529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:26.370610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:26.371411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 152/312] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:26.378696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:26.411934 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:26.416767 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b96d8-30d3-45ab-9330-f4c424f3ee82 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:26.461420 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:26.462689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:26.463721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:26.464423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] PATH_INFO: `/v3/system/users/8846c7627dd04387be52adc1c67b37ef/roles/bddb19f21b7f472589b9154cb2aeeecc` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:26.466185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8846c7627dd04387be52adc1c67b37ef, role_id=bddb19f21b7f472589b9154cb2aeeecc)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:26.468968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:26.509406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266210c-e34d-4d73-88ab-ac573e9cc3ff None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:26.549064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:26.549708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:26.550050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:26.550325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:26.550987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:26.551292 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:26.552695 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:26.562010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 153/313] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:26.567901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:26.583035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 161/314] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Sep 1 11:57:26 2022] PUT /identity/v3/system/users/8846c7627dd04387be52adc1c67b37ef/roles/bddb19f21b7f472589b9154cb2aeeecc => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:57:26.678391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:26.692977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14a4ae-705b-41fc-b9a8-191aa2af93f7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:26.733568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:26.734287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:26.734571 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:26.736443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] PATH_INFO: `/v3/users/24d0197fe7f14384b6b829c27e6e17f6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:26.736443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] RBAC: Authorizing `identity:get_user(user_id=24d0197fe7f14384b6b829c27e6e17f6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:26.737717 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:26.739817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 154/315] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:26 2022] GET /identity/v3/users/24d0197fe7f14384b6b829c27e6e17f6 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:26.745791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:26.840231 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:26.884995 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b853bab-7b71-4fce-b740-6f2bd632f089 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:26.952268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:26.954502 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:26.955214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:26.955919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:27.096758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:27.100201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:27.259886 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:27.298419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 162/316] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:27.304503 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:27.421001 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a1167-86c9-4aca-8ce7-bfdf65d9ba47 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:27.462025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5da4f-e3ed-41e5-a47e-6d476733d3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:27.462701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5da4f-e3ed-41e5-a47e-6d476733d3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:27.462960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5da4f-e3ed-41e5-a47e-6d476733d3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:27.463227 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5da4f-e3ed-41e5-a47e-6d476733d3d2 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:27.464886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5da4f-e3ed-41e5-a47e-6d476733d3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:27.466459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5da4f-e3ed-41e5-a47e-6d476733d3d2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:27.469151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 155/317] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:27 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:27.475375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:27.585257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748e21b-2c98-49d3-a0bd-bfe39c429438 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:27.622373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a25201-b2f1-4a5e-bbae-a4ae2b302734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:27.622988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a25201-b2f1-4a5e-bbae-a4ae2b302734 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:27.623271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a25201-b2f1-4a5e-bbae-a4ae2b302734 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:27.623541 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a25201-b2f1-4a5e-bbae-a4ae2b302734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:27.624264 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a25201-b2f1-4a5e-bbae-a4ae2b302734 None admin] RBAC: Adding query filter params (role.id=79a475679b454999b9d77c6405a7f0a7, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a, user.id=24d0197fe7f14384b6b829c27e6e17f6) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:27.624578 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a25201-b2f1-4a5e-bbae-a4ae2b302734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:27.626053 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a25201-b2f1-4a5e-bbae-a4ae2b302734 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:27.632482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 163/318] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:57:27 2022] GET /identity/v3/role_assignments?user.id=24d0197fe7f14384b6b829c27e6e17f6&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=79a475679b454999b9d77c6405a7f0a7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:28.300789 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0e762-5fae-4cd1-84e9-edfdafb338c5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:28.301067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0e762-5fae-4cd1-84e9-edfdafb338c5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:28.301484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0e762-5fae-4cd1-84e9-edfdafb338c5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:28.302976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 156/319] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:28.308267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbbb76-b0c8-407d-b849-b1e1e4caa70b None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:28.308935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbbb76-b0c8-407d-b849-b1e1e4caa70b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:28.308935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbbb76-b0c8-407d-b849-b1e1e4caa70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:28.310895 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fb9c-ea3d-46d2-bdd9-35696fdd8c3e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:28.311363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fb9c-ea3d-46d2-bdd9-35696fdd8c3e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:28.311813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fb9c-ea3d-46d2-bdd9-35696fdd8c3e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:28.313906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 157/320] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:28.320938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974fc28-6ef1-4c0a-8f51-c46e390e351e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:28.321300 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974fc28-6ef1-4c0a-8f51-c46e390e351e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:28.321763 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974fc28-6ef1-4c0a-8f51-c46e390e351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:28.330907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-5fbbbb76-b0c8-407d-b849-b1e1e4caa70b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:28.330907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-5fbbbb76-b0c8-407d-b849-b1e1e4caa70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:28.352573 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7974fc28-6ef1-4c0a-8f51-c46e390e351e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:28.354642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7974fc28-6ef1-4c0a-8f51-c46e390e351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:28.369895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:28.381079 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:28.408837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 164/321] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:28.452536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 158/322] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:28.722241 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbbb76-b0c8-407d-b849-b1e1e4caa70b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:28.744852 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974fc28-6ef1-4c0a-8f51-c46e390e351e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:28.863854 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbbb76-b0c8-407d-b849-b1e1e4caa70b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:28.881069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974fc28-6ef1-4c0a-8f51-c46e390e351e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:28.922730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:28.923832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:28.924338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:28.924834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:28.925768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:28.926682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:28.927105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:28.927539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:28.928858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:28.930574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:28.932693 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 159/323] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:28.939579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:28.945515 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:28.955388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:28.957195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:28.958704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 165/324] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:28.962778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:28.979111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:29.008570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:29.102415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409bdc5-b025-4f4b-91e1-1073860d3662 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:29.120686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adc94c-970f-49de-b54f-08a98680b0b3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:29.144248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21162c-5422-4851-8494-ca4302d5a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:29.144870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21162c-5422-4851-8494-ca4302d5a2b4 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:29.145151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21162c-5422-4851-8494-ca4302d5a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:29.145425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21162c-5422-4851-8494-ca4302d5a2b4 None admin] PATH_INFO: `/v3/groups` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:29.146104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21162c-5422-4851-8494-ca4302d5a2b4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:29.147599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21162c-5422-4851-8494-ca4302d5a2b4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:29.167504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c35368-f8f7-4235-a500-a4285afd9b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:29.170008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c35368-f8f7-4235-a500-a4285afd9b69 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:29.170008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c35368-f8f7-4235-a500-a4285afd9b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:29.170008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c35368-f8f7-4235-a500-a4285afd9b69 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:29.170008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c35368-f8f7-4235-a500-a4285afd9b69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:29.170008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c35368-f8f7-4235-a500-a4285afd9b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:29.170983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c35368-f8f7-4235-a500-a4285afd9b69 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:29.179918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 166/325] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:29.181175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df5f14-c9c8-4db4-acee-d2f563b281be None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:29.181471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df5f14-c9c8-4db4-acee-d2f563b281be None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:29.181777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df5f14-c9c8-4db4-acee-d2f563b281be None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:29.183233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 167/326] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:29.186701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df5f14-c9c8-4db4-acee-d2f563b281be None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:29.271457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 160/327] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Sep 1 11:57:28 2022] POST /identity/v3/groups => generated 226 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:57:29.272677 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27a5cb-7b4c-4132-86ba-20ddfadff286 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:29.272965 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27a5cb-7b4c-4132-86ba-20ddfadff286 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:29.273240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27a5cb-7b4c-4132-86ba-20ddfadff286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:29.303642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:29.313191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8d27a5cb-7b4c-4132-86ba-20ddfadff286 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:29.315592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8d27a5cb-7b4c-4132-86ba-20ddfadff286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:29.316106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df5f14-c9c8-4db4-acee-d2f563b281be None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:29.387310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:29.388208 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:29.388631 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:29.389057 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] PATH_INFO: `/v3/users/7db38f74a07b4e2eb095fed2c95e5fe2` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:29.390836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] RBAC: Authorizing `identity:get_user(user_id=7db38f74a07b4e2eb095fed2c95e5fe2)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:29.392880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:29.399854 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 168/328] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:29 2022] GET /identity/v3/users/7db38f74a07b4e2eb095fed2c95e5fe2 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:29.401537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:29.433697 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 161/329] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:29.557165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1de6-d23e-43d4-aea8-b01ab0932904 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:29.607567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:29.608483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:29.608957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:29.610173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] PATH_INFO: `/v3/system/users/7db38f74a07b4e2eb095fed2c95e5fe2/roles/79a475679b454999b9d77c6405a7f0a7` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:29.612841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7db38f74a07b4e2eb095fed2c95e5fe2, role_id=79a475679b454999b9d77c6405a7f0a7)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:29.614453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:29.677940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 169/330] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Sep 1 11:57:29 2022] PUT /identity/v3/system/users/7db38f74a07b4e2eb095fed2c95e5fe2/roles/79a475679b454999b9d77c6405a7f0a7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:29.712699 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27a5cb-7b4c-4132-86ba-20ddfadff286 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:29.726431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:29.860177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27a5cb-7b4c-4132-86ba-20ddfadff286 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:29.912760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:29.913866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:29.914356 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:29.914828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:29.928406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:29.934025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:29.934977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:29.935771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:29.938613 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 162/331] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:29.944997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:29.997093 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:30.119122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb877a42-d1fb-4a27-ac32-7047cf212751 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:30.200103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:30.201452 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:30.202185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:30.202856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:30.204677 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:30.205388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:30.208014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:30.228915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 170/332] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:30.234988 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:30.340884 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:30.371621 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ed5c3-8fd6-4b08-b4b4-5c3f3b9d68bd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:30.414043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375f608-0a10-4aa5-a793-9b14a8050e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:30.414690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f608-0a10-4aa5-a793-9b14a8050e45 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:30.414967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f608-0a10-4aa5-a793-9b14a8050e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:30.415243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f608-0a10-4aa5-a793-9b14a8050e45 None admin] PATH_INFO: `/v3/users/8846c7627dd04387be52adc1c67b37ef` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:30.416438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f608-0a10-4aa5-a793-9b14a8050e45 None admin] RBAC: Authorizing `identity:get_user(user_id=8846c7627dd04387be52adc1c67b37ef)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:30.417933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f608-0a10-4aa5-a793-9b14a8050e45 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:30.419727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 163/333] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:30 2022] GET /identity/v3/users/8846c7627dd04387be52adc1c67b37ef => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:30.426684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:30.568591 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:30.590376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980cd6-5232-4b5c-b979-af1eaf88f0c0 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:30.637051 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760fea-2a45-4d0c-a9b6-a8ed4d5f9d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:30.637579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760fea-2a45-4d0c-a9b6-a8ed4d5f9d2a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:30.637886 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760fea-2a45-4d0c-a9b6-a8ed4d5f9d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:30.638163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760fea-2a45-4d0c-a9b6-a8ed4d5f9d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:30.638898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33760fea-2a45-4d0c-a9b6-a8ed4d5f9d2a None admin] RBAC: Adding query filter params (role.id=bddb19f21b7f472589b9154cb2aeeecc, user.id=8846c7627dd04387be52adc1c67b37ef, scope.system=all) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:30.639214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33760fea-2a45-4d0c-a9b6-a8ed4d5f9d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:30.640708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33760fea-2a45-4d0c-a9b6-a8ed4d5f9d2a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:30.657368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 171/334] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Sep 1 11:57:30 2022] GET /identity/v3/role_assignments?user.id=8846c7627dd04387be52adc1c67b37ef&scope.system=all&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:31.897217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373f3b8-6d16-4ae5-827d-be29a588b972 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:31.898027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373f3b8-6d16-4ae5-827d-be29a588b972 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:31.898027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373f3b8-6d16-4ae5-827d-be29a588b972 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:31.899401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 164/335] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:31.908854 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f56f5-1eb2-4565-b374-cf3bc17d0a30 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:31.909876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f56f5-1eb2-4565-b374-cf3bc17d0a30 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:31.909876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f56f5-1eb2-4565-b374-cf3bc17d0a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:31.951222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-c01f56f5-1eb2-4565-b374-cf3bc17d0a30 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:31.953912 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-c01f56f5-1eb2-4565-b374-cf3bc17d0a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:32.063984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 172/336] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:32.626878 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373f3b8-6d16-4ae5-827d-be29a588b972 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:32.849973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373f3b8-6d16-4ae5-827d-be29a588b972 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:32.897954 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca3742-53b4-460b-a18b-c1bf515722de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:32.898587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca3742-53b4-460b-a18b-c1bf515722de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:32.898866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca3742-53b4-460b-a18b-c1bf515722de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:32.899144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca3742-53b4-460b-a18b-c1bf515722de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:32.904137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:32.913743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca3742-53b4-460b-a18b-c1bf515722de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-15ca3742-53b4-460b-a18b-c1bf515722de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:32.924089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:32.925312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:32.926170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 165/337] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:32.933353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f56f5-1eb2-4565-b374-cf3bc17d0a30 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:33.030025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:33.062184 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f56f5-1eb2-4565-b374-cf3bc17d0a30 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:33.111138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:33.111757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.112045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.112322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.113012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:33.113328 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:33.114752 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:33.128243 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 173/338] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:33.134092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:33.134368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63dca3-d22c-4b22-9fce-6e223b4a66ac None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.134846 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63dca3-d22c-4b22-9fce-6e223b4a66ac None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.135463 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63dca3-d22c-4b22-9fce-6e223b4a66ac None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.141149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 166/339] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:33.150197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b604-4b03-4286-9a74-00efd3f7fe0b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.151073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b604-4b03-4286-9a74-00efd3f7fe0b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.151757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b604-4b03-4286-9a74-00efd3f7fe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.198648 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ede7b604-4b03-4286-9a74-00efd3f7fe0b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:33.209642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ede7b604-4b03-4286-9a74-00efd3f7fe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:33.274196 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191c424-69c8-463e-91a1-0eeab4d34e44 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:33.317242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:33.317909 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.318210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.318472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.327618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:33.366022 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 167/340] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:33.367231 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f93c9-9db5-4ce9-8d12-a5b0da62e17a None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.367519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f93c9-9db5-4ce9-8d12-a5b0da62e17a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.367796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f93c9-9db5-4ce9-8d12-a5b0da62e17a None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.369249 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 168/341] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:33.374943 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c470c6-7052-40d8-9b3b-3655e83f3fc5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.375443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c470c6-7052-40d8-9b3b-3655e83f3fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.375860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c470c6-7052-40d8-9b3b-3655e83f3fc5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.377380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 169/342] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:33.383479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0878b-3e5c-4002-8b0f-44154633c8fc None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.383981 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0878b-3e5c-4002-8b0f-44154633c8fc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.384403 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0878b-3e5c-4002-8b0f-44154633c8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.415514 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-32a0878b-3e5c-4002-8b0f-44154633c8fc None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:33.417018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-32a0878b-3e5c-4002-8b0f-44154633c8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:33.418582 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:33.420244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:33.539382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 170/343] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:33.539628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 174/344] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:57:33.541867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66a8e0-2fca-409b-8fe3-aea1c8346182 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.542159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66a8e0-2fca-409b-8fe3-aea1c8346182 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.542436 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66a8e0-2fca-409b-8fe3-aea1c8346182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.545457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:33.578843 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1f66a8e0-2fca-409b-8fe3-aea1c8346182 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:33.580068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1f66a8e0-2fca-409b-8fe3-aea1c8346182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:33.584732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:33.659859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984fa57-c81d-4f95-b8f2-25436301e820 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:33.674226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:33.687238 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 171/345] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:33.711087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:33.711755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.712041 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.712317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.715321 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:33.716793 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:33.719081 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 175/346] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:33 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:33.725045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66a8e0-2fca-409b-8fe3-aea1c8346182 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:33.859854 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66a8e0-2fca-409b-8fe3-aea1c8346182 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:33.908901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:33.910439 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:33.910870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:33.911320 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:33.921688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:33.923459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:33.925842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 172/347] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:33 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:33.931618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:34.078223 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490adcd-11a8-4bde-9094-e206bee81f1b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:34.081019 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:34.118737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:34.128860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:34.129218 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:34.129422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:34.129726 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:34.130489 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:34.130919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:34.132411 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:34.139556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 176/348] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:33 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:34.218279 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:34.252114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:34.354257 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e82f1-0a64-4a14-9b05-330f59b8c7cb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:34.500500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:34.501580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:34.502097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:34.502579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:34.543140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:34.556973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:34.557976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:34.558782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:34.560146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 173/349] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:34.561661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:34.581676 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db01ad6-f8c4-45f9-8674-032131d99e0c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:34.661200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:34.662325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:34.662919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:34.663377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:34.684780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:34.696342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:34.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:34.698172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:34.699394 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 177/350] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:34.700539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:34.750263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c4a30-0a18-4534-a9b7-abc42e47a792 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:34.783791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:34.793868 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:34.879800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:34.879800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:34.879800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:34.880385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:34.907563 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:34.918626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:34.920046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:34.920845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 174/351] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:34.922448 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:34.962094 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6487cdd-98a4-454c-bca7-72cb6ae24831 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:35.005243 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:35.005921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.006209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.006485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.007165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:35.007541 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:35.009106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:35.020497 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 178/352] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:35.025784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:35.125167 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21959723-d738-4113-b7e9-31edd54f5705 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:35.162895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:35.175493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc2499-39a0-40d7-bd53-c1d8b87a4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:35.176817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc2499-39a0-40d7-bd53-c1d8b87a4c67 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.178245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc2499-39a0-40d7-bd53-c1d8b87a4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.178376 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc2499-39a0-40d7-bd53-c1d8b87a4c67 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.179804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc2499-39a0-40d7-bd53-c1d8b87a4c67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:35.188418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc2499-39a0-40d7-bd53-c1d8b87a4c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:35.188554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bb644-b8bd-4d80-a068-a54f16666449 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:35.188648 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc2499-39a0-40d7-bd53-c1d8b87a4c67 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:35.204258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 175/353] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:35.204258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be682f-3bef-43ef-b782-5c19a621fce3 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.204258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be682f-3bef-43ef-b782-5c19a621fce3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.204258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be682f-3bef-43ef-b782-5c19a621fce3 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.204681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 176/354] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:35.206509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be682f-3bef-43ef-b782-5c19a621fce3 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:35.251925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:35.255153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7ef34-0399-4233-9ca3-af45ca95105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:35.255822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7ef34-0399-4233-9ca3-af45ca95105f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.256109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7ef34-0399-4233-9ca3-af45ca95105f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.256392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7ef34-0399-4233-9ca3-af45ca95105f None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.257095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7ef34-0399-4233-9ca3-af45ca95105f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:35.257406 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7ef34-0399-4233-9ca3-af45ca95105f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:35.258860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7ef34-0399-4233-9ca3-af45ca95105f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:35.279415 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 179/355] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:35.282281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f81fb-bdf0-40e4-9387-506fe6d1a0c6 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.283085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f81fb-bdf0-40e4-9387-506fe6d1a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.284128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f81fb-bdf0-40e4-9387-506fe6d1a0c6 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.287561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 180/356] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:35.290112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dae8f2-d2a4-4dc8-bd3a-953f364d9150 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.290919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dae8f2-d2a4-4dc8-bd3a-953f364d9150 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.291640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dae8f2-d2a4-4dc8-bd3a-953f364d9150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.325535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-88dae8f2-d2a4-4dc8-bd3a-953f364d9150 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:35.327753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-88dae8f2-d2a4-4dc8-bd3a-953f364d9150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:35.386735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be682f-3bef-43ef-b782-5c19a621fce3 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:35.443179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:35.443443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.443574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.443922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.469221 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 181/357] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:35.471113 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dae8f2-d2a4-4dc8-bd3a-953f364d9150 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:35.527480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:35.528994 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:35.567637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:35.665496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 177/358] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:35.666422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:35.670312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:35.718103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dae8f2-d2a4-4dc8-bd3a-953f364d9150 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:35.790655 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa99d6-a54f-4d23-b055-3ff58cb68671 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:35.792463 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:35.793689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.794175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.794647 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.829054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:35.839269 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296e9eb-d580-4462-932c-f215b1ebf266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:35.839924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296e9eb-d580-4462-932c-f215b1ebf266 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.840204 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296e9eb-d580-4462-932c-f215b1ebf266 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.840478 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296e9eb-d580-4462-932c-f215b1ebf266 None admin] PATH_INFO: `/v3/users/7db38f74a07b4e2eb095fed2c95e5fe2` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.841842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296e9eb-d580-4462-932c-f215b1ebf266 None admin] RBAC: Authorizing `identity:get_user(user_id=7db38f74a07b4e2eb095fed2c95e5fe2)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:35.843347 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296e9eb-d580-4462-932c-f215b1ebf266 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:35.845064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 178/359] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:57:35 2022] GET /identity/v3/users/7db38f74a07b4e2eb095fed2c95e5fe2 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:35.846444 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1b50-a279-4dac-8907-df7a5d5e5a02 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:35.846732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1b50-a279-4dac-8907-df7a5d5e5a02 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:35.849414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1b50-a279-4dac-8907-df7a5d5e5a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:35.874551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-aa7e1b50-a279-4dac-8907-df7a5d5e5a02 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:35.877309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-aa7e1b50-a279-4dac-8907-df7a5d5e5a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:35.908783 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:35.911840 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:36.030016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 179/360] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:36.034231 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1b50-a279-4dac-8907-df7a5d5e5a02 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:36.043854 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:36.079465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 182/361] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:36.080671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:36.211325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a169b16-5c5c-4f68-946b-68a5bfe4c490 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:36.252562 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:36.253395 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:36.253806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:36.254086 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:36.254838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] RBAC: Adding query filter params (role.id=79a475679b454999b9d77c6405a7f0a7, user.id=7db38f74a07b4e2eb095fed2c95e5fe2, scope.system=all) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:36.255147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:36.256590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:36.277644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 183/362] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Sep 1 11:57:36 2022] GET /identity/v3/role_assignments?user.id=7db38f74a07b4e2eb095fed2c95e5fe2&scope.system=all&role.id=79a475679b454999b9d77c6405a7f0a7 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:36.277644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:36.304114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1b50-a279-4dac-8907-df7a5d5e5a02 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:36.342258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:36.388575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:36.390882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:36.391759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:36.392515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:36.395212 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:36.398386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:36.401812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 180/363] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:36 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:36.403760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:36.411767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:36.420306 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2cd95-dade-4bf6-b662-f1e3cfeac2e1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:36.470480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:36.471156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:36.471438 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:36.471722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:36.487062 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:36.493769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:36.495545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 184/364] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:36.497039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:36.658685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41259f50-ec17-4f31-9b91-2035e7385b1a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:36.712301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:36.718984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5d7ff-d54e-445e-9fcf-7115c613fc5c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:36.725316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:36.727579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:36.727579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:36.727768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:36.729835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:36.732860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:36.738417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 181/365] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:36 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:36.739542 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:36.788029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:36.789126 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:36.789676 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:36.790225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:36.812298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:36.820527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:36.822364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:36.824032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:36.827219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 185/366] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:57:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:36.829528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:36.876770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:36.947371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe444529-ed17-4886-ab22-13a9b7bc943e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:37.019687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:37.020708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.021170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.021671 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.024506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:37.027942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:37.030290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 182/367] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:36 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:37.031796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:37.049065 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe344-d01e-4eaa-9dcf-97ac79233db1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:37.097060 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:37.098190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.098477 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.098748 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.099630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:37.100063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:37.101504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:37.114248 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 186/368] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:37.115155 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:37.146998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:37.255237 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7067b9-3167-479c-bd83-0fd602b28c58 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:37.271695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0740c6-1332-4127-b3d1-59cadce5ea4f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:37.308211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:37.309838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.309838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.312866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.312866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:37.315386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:37.318539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 183/369] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:37 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:37.320481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:37.325645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:37.334018 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:37.340364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.340364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.340364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.340364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:37.340364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:37.340364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:37.349478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 187/370] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:37.350395 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:37.496332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:37.523132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e4bc3-50d4-4023-a41c-e139e1e2f4ed None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:37.531477 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f8229-f12f-44c3-8f77-1abbc1d20a5e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:37.580776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5db5f-ff44-4566-a02e-785c4d7c0975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:37.582432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5db5f-ff44-4566-a02e-785c4d7c0975 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.582923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5db5f-ff44-4566-a02e-785c4d7c0975 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.583393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5db5f-ff44-4566-a02e-785c4d7c0975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.584634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5db5f-ff44-4566-a02e-785c4d7c0975 None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:37.585172 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5db5f-ff44-4566-a02e-785c4d7c0975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:37.587600 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee9fb4-011d-4b3a-87be-d9f36bfd5495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:37.588027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5db5f-ff44-4566-a02e-785c4d7c0975 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:37.588348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9fb4-011d-4b3a-87be-d9f36bfd5495 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.588679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9fb4-011d-4b3a-87be-d9f36bfd5495 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.589005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9fb4-011d-4b3a-87be-d9f36bfd5495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.600415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 184/371] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:37 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:37.602646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77d102-9cf3-47ed-bd28-0df2b75e68a4 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.603106 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77d102-9cf3-47ed-bd28-0df2b75e68a4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.603598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77d102-9cf3-47ed-bd28-0df2b75e68a4 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.606500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 185/372] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:37.607766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77d102-9cf3-47ed-bd28-0df2b75e68a4 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:37.734967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee9fb4-011d-4b3a-87be-d9f36bfd5495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:37.737914 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee9fb4-011d-4b3a-87be-d9f36bfd5495 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:37.758411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:37.845582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77d102-9cf3-47ed-bd28-0df2b75e68a4 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:37.895538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:37.896175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.896459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.896769 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.897565 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:37.897946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:37.899647 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:37.907195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 186/373] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:37 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 260 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:37.908321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:37.911013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:37.940175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 188/374] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:37.941849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288889f2-fa33-40d4-9989-459e50728555 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:37.943012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288889f2-fa33-40d4-9989-459e50728555 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:37.943494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288889f2-fa33-40d4-9989-459e50728555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:37.972891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-288889f2-fa33-40d4-9989-459e50728555 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:37.976652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-288889f2-fa33-40d4-9989-459e50728555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:38.058490 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a5a7d-952d-462d-aa35-b673f7450b5e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:38.134431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:38.134728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 189/375] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:38.136609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288889f2-fa33-40d4-9989-459e50728555 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:38.155845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:38.157364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:38.158148 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:38.159042 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:38.300976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:38.304117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:38.308673 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288889f2-fa33-40d4-9989-459e50728555 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:38.342442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:38.352610 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:38.353661 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:38.354427 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:38.354920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:38.356978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:38.359053 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:38.361720 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 190/376] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:38 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:38.391089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:38.618459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 187/377] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:38.627082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:38.639251 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5127a-11c7-4db2-a028-9a1da93b5aa4 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:38.682998 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:38.687742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:38.688844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:38.689334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:38.689955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:38.692219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:38.695417 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:38.699211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 191/378] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:38 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:38.700922 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:38.808813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:38.828846 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0527913-428e-4a4f-b875-395407f67bc7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:38.885532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9ddcc-dfbf-44ef-b938-ad851df81356 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:38.892177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:38.892825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:38.893103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:38.893378 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:38.905569 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:38.911034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:38.913059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:38.914930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:38.917054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 188/379] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:38.918056 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:38.964180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:38.965822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:38.966529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:38.967208 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] PATH_INFO: `/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:38.969926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] RBAC: Authorizing `identity:get_project(project_id=d6c17f90daea4fc3b1b162a692b4ec70)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:38.973140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:38.983879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 192/380] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:38 2022] GET /identity/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70 => generated 344 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:38.986061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:39.026085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:39.038586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980d2af-a4ca-48c1-b1e8-b815a5a058ee None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:39.082975 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:39.083955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:39.083955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:39.084265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:39.084852 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:39.085161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:39.086950 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:39.094166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 189/381] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:38 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:39.095061 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:39.208803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c27240-0a52-4ebd-9864-4bc118f429be None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:39.258048 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:39.286831 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:39.288281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:39.288987 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:39.295447 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:39.297070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:39.297858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:39.300829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:39.333017 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 193/382] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:39.340965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:39.349645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47372002-a225-4530-b11d-d25d8ad65abf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:39.456100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:39.456784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:39.458600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:39.458949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] PATH_INFO: `/v3/groups/083e2d9a0bc74297ad45d3e9b115cc0c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:39.470598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] RBAC: Authorizing `identity:get_group(group_id=083e2d9a0bc74297ad45d3e9b115cc0c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:39.472155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:39.475508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 190/383] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:39 2022] GET /identity/v3/groups/083e2d9a0bc74297ad45d3e9b115cc0c => generated 229 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:39.485103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:39.525181 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:39.583098 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4c7d-e710-4500-843a-0c1f9ce75c5b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:39.648405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f81720-83c1-4a10-b80d-9d4e3c8d7f81 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:39.686540 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:39.687729 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:39.688013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:39.688293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:39.689422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:39.690076 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:39.690331 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:39.690484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:39.690616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:39.691331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] RBAC: Adding query filter params (group.id=083e2d9a0bc74297ad45d3e9b115cc0c, role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=d6c17f90daea4fc3b1b162a692b4ec70) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:39.691637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:39.691879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:39.693072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:39.694079 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 194/384] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:39 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:39.705170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 191/385] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:39 2022] GET /identity/v3/role_assignments?group.id=083e2d9a0bc74297ad45d3e9b115cc0c&scope.project.id=d6c17f90daea4fc3b1b162a692b4ec70&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:39.706786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:39.786619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:39.892478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceb430-36d7-42f6-ad85-d288dc086661 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:39.971928 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ff8a8-cfb8-4f1c-90d7-bca213a74636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:39.973520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff8a8-cfb8-4f1c-90d7-bca213a74636 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:39.974377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff8a8-cfb8-4f1c-90d7-bca213a74636 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:39.975138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff8a8-cfb8-4f1c-90d7-bca213a74636 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:39.983029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ff8a8-cfb8-4f1c-90d7-bca213a74636 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:39.986295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ff8a8-cfb8-4f1c-90d7-bca213a74636 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:39.989761 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 195/386] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:39 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:39.996740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:40.204954 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:40.208485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbfd0f-59fd-4d9a-97b4-151b1e15b291 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:40.209337 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbfd0f-59fd-4d9a-97b4-151b1e15b291 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:40.209786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbfd0f-59fd-4d9a-97b4-151b1e15b291 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:40.211785 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 196/387] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:40.219549 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4ba24-33b7-4d87-bfb9-f323bcf3ec19 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:40.220029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4ba24-33b7-4d87-bfb9-f323bcf3ec19 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:40.220509 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4ba24-33b7-4d87-bfb9-f323bcf3ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:40.230317 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7fa1-4ec2-4330-b6d6-87b9f51a6c22 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:40.259181 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-9bf4ba24-33b7-4d87-bfb9-f323bcf3ec19 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:40.260429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9bf4ba24-33b7-4d87-bfb9-f323bcf3ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:40.307611 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:40.308378 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:40.308667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:40.308986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/bddb19f21b7f472589b9154cb2aeeecc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:40.322526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba34f3047cae4d5d8da4129e3b26f8b3, group_id=d4255c975d8b42a3bcb63bbbe285a5e9, role_id=bddb19f21b7f472589b9154cb2aeeecc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:40.324022 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:40.359408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 197/388] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:40.411725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 192/389] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Sep 1 11:57:39 2022] PUT /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/bddb19f21b7f472589b9154cb2aeeecc => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:40.672888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4ba24-33b7-4d87-bfb9-f323bcf3ec19 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:40.729717 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:40.816190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4ba24-33b7-4d87-bfb9-f323bcf3ec19 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:40.863520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:40.864215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:40.864667 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:40.865098 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:40.884516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:40.894179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:40.895852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:40.897341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 198/390] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:40.907143 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:41.107728 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9b19-13a0-4bd8-98c9-25831644f492 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:41.169149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:41.170211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:41.170671 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:41.171180 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:41.172193 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:41.172547 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:41.175313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:41.190989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 193/391] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:41.200272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:41.430578 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6735a9f-c8e1-46f7-9c8d-933cc747c02a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:41.448699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:41.507101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:41.508222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:41.508826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:41.509312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:41.511628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:41.514441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:41.518635 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 199/392] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:41 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:41.527566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:41.534745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:41.758445 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c28c-ae3b-4caa-be10-b20b34a33995 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:41.825839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23ceb2-b3e3-4812-9746-9da50e121e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:41.826996 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23ceb2-b3e3-4812-9746-9da50e121e3d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:41.827476 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23ceb2-b3e3-4812-9746-9da50e121e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:41.828515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23ceb2-b3e3-4812-9746-9da50e121e3d None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:41.830649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23ceb2-b3e3-4812-9746-9da50e121e3d None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:41.833396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23ceb2-b3e3-4812-9746-9da50e121e3d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:41.836453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 194/393] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:41 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:41.845768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:42.005043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9fc38-4e53-4373-b8b2-986a8beefc9b None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:42.005566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9fc38-4e53-4373-b8b2-986a8beefc9b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:42.006046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9fc38-4e53-4373-b8b2-986a8beefc9b None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:42.008783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 195/394] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:42.018796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278d96-fecc-4a99-a02a-4461d241f3bd None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:42.019251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278d96-fecc-4a99-a02a-4461d241f3bd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:42.019489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278d96-fecc-4a99-a02a-4461d241f3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:42.077056 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ac278d96-fecc-4a99-a02a-4461d241f3bd None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:42.079949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ac278d96-fecc-4a99-a02a-4461d241f3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:42.110772 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461a7f6-e908-4ca7-b218-f21a312497bc None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:42.212674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32576ce-7b0c-4e3f-9410-6e3003717f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:42.213969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32576ce-7b0c-4e3f-9410-6e3003717f2c None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:42.214466 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32576ce-7b0c-4e3f-9410-6e3003717f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:42.214957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32576ce-7b0c-4e3f-9410-6e3003717f2c None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/bddb19f21b7f472589b9154cb2aeeecc` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:42.221277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32576ce-7b0c-4e3f-9410-6e3003717f2c None admin] RBAC: Authorizing `identity:create_grant(project_id=8b1e5e0ec4174aac9379046a27e7794a, group_id=d4255c975d8b42a3bcb63bbbe285a5e9, role_id=bddb19f21b7f472589b9154cb2aeeecc)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:42.224345 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32576ce-7b0c-4e3f-9410-6e3003717f2c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:42.295242 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 196/395] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:42.327445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 200/396] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Sep 1 11:57:41 2022] PUT /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/bddb19f21b7f472589b9154cb2aeeecc => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:42.582415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143e4e4-b7e3-4ba9-943f-97121ad60d43 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:42.583186 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143e4e4-b7e3-4ba9-943f-97121ad60d43 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:42.583186 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143e4e4-b7e3-4ba9-943f-97121ad60d43 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:42.585775 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 197/397] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:42.595202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3e838-2656-4895-8981-58eceabe23c6 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:42.595958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3e838-2656-4895-8981-58eceabe23c6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:42.596125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3e838-2656-4895-8981-58eceabe23c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:42.629508 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-e2f3e838-2656-4895-8981-58eceabe23c6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:42.630785 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-e2f3e838-2656-4895-8981-58eceabe23c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:42.667473 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:42.686205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143e4e4-b7e3-4ba9-943f-97121ad60d43 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:42.691576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:42.752359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 201/398] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:42.889295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143e4e4-b7e3-4ba9-943f-97121ad60d43 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:42.951330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6c774-a07f-4d9d-9fae-037ff6a37882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:42.951998 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6c774-a07f-4d9d-9fae-037ff6a37882 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:42.952278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6c774-a07f-4d9d-9fae-037ff6a37882 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:42.952552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6c774-a07f-4d9d-9fae-037ff6a37882 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:42.966214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6c774-a07f-4d9d-9fae-037ff6a37882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-55e6c774-a07f-4d9d-9fae-037ff6a37882 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:42.972190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:42.974129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:42.975766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 198/399] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:42.987333 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3e838-2656-4895-8981-58eceabe23c6 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.175205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3e838-2656-4895-8981-58eceabe23c6 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:43.208681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34ac36-6f2c-4e40-bad7-93d7e68ef5e2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.209720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34ac36-6f2c-4e40-bad7-93d7e68ef5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.210495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34ac36-6f2c-4e40-bad7-93d7e68ef5e2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.213729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 199/400] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:43.215580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34ac36-6f2c-4e40-bad7-93d7e68ef5e2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.255639 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902b6b3-fbce-415f-8653-9a6169bad624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:43.257436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902b6b3-fbce-415f-8653-9a6169bad624 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.258246 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902b6b3-fbce-415f-8653-9a6169bad624 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.259018 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902b6b3-fbce-415f-8653-9a6169bad624 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.261884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902b6b3-fbce-415f-8653-9a6169bad624 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:43.262221 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902b6b3-fbce-415f-8653-9a6169bad624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:43.263650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902b6b3-fbce-415f-8653-9a6169bad624 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:43.280827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 202/401] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:43.283750 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09611-7fc9-4f45-9b65-320e6b5b72c6 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.284567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09611-7fc9-4f45-9b65-320e6b5b72c6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.285395 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09611-7fc9-4f45-9b65-320e6b5b72c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.295794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:43.316910 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-4cb09611-7fc9-4f45-9b65-320e6b5b72c6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:43.319144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-4cb09611-7fc9-4f45-9b65-320e6b5b72c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:43.344609 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34ac36-6f2c-4e40-bad7-93d7e68ef5e2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:43.360431 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:43.387283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bab04-c365-40b6-a35b-88314a0f34ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:43.388011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bab04-c365-40b6-a35b-88314a0f34ab None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.388298 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bab04-c365-40b6-a35b-88314a0f34ab None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.388569 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bab04-c365-40b6-a35b-88314a0f34ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.402891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bab04-c365-40b6-a35b-88314a0f34ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5b1bab04-c365-40b6-a35b-88314a0f34ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:43.409039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:43.410107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:43.410901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:43.415171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 200/402] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:43.416609 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be8631-801b-4c17-9292-b35c181f6143 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.416897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be8631-801b-4c17-9292-b35c181f6143 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.417173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be8631-801b-4c17-9292-b35c181f6143 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.423461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 201/403] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:43.423461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be8631-801b-4c17-9292-b35c181f6143 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.429021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 203/404] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:43.430256 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb09611-7fc9-4f45-9b65-320e6b5b72c6 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.549634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb09611-7fc9-4f45-9b65-320e6b5b72c6 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:43.585001 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be8631-801b-4c17-9292-b35c181f6143 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:43.589436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66c00b-d379-49c4-bb1e-a772355e5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:43.590343 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66c00b-d379-49c4-bb1e-a772355e5809 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.590766 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66c00b-d379-49c4-bb1e-a772355e5809 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.591205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66c00b-d379-49c4-bb1e-a772355e5809 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.592039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66c00b-d379-49c4-bb1e-a772355e5809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:43.592504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66c00b-d379-49c4-bb1e-a772355e5809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:43.594795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66c00b-d379-49c4-bb1e-a772355e5809 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:43.604646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 204/405] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:43.606899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1038fb-6f9b-45d9-ac36-7c635bdf1ac3 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.607421 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1038fb-6f9b-45d9-ac36-7c635bdf1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.607888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1038fb-6f9b-45d9-ac36-7c635bdf1ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.620014 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:43.629014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-7d1038fb-6f9b-45d9-ac36-7c635bdf1ac3 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:43.637542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7d1038fb-6f9b-45d9-ac36-7c635bdf1ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:43.637718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:43.637718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.637718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.637718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.638239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:43.639820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:43.647357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 202/406] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:43 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:43.647357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.653496 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:43.721912 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 205/407] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:43.723350 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1038fb-6f9b-45d9-ac36-7c635bdf1ac3 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.771028 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6acee3-b4ed-4f60-a5e5-d7f9a19b2c0e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:43.813630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:43.813630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.813630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.813630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.814061 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:43.815176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:43.817539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 203/408] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:43 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:43.823473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.888153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1038fb-6f9b-45d9-ac36-7c635bdf1ac3 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:43.891584 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:43.929077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:43.930147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.930628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.931097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.932499 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:43.934205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:43.936122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 206/409] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:43 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:43.942332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c96281-8c80-4039-a9a8-5e5881c20067 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:43.942579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:43.963209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:43.984424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:43.985705 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:43.986059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:43.986406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:43.987682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:43.989293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:43.991471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 204/410] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:43 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:43.992392 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:44.058994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa92cb-d92f-49c1-a05b-318f2e8a2721 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.097532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:44.098613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:44.099082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:44.099566 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:44.109741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d67714-5103-4d14-a662-21ac9896a5f8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.112822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:44.118348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:44.119318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:44.120130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:44.122754 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 207/411] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:57:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:44.124034 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:44.150462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:44.162123 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:44.164585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:44.164585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:44.164585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:44.166201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba34f3047cae4d5d8da4129e3b26f8b3, group_id=d4255c975d8b42a3bcb63bbbe285a5e9, role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:44.167937 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:44.226596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:44.253663 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc242d-fff3-46be-bc38-8105d50a3ba9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.264650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 205/412] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Sep 1 11:57:43 2022] PUT /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:44.278622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:44.300815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:44.301892 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:44.302356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:44.302791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:44.306212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] RBAC: Authorizing `identity:create_grant(project_id=8b1e5e0ec4174aac9379046a27e7794a, group_id=d4255c975d8b42a3bcb63bbbe285a5e9, role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:44.308027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:44.410732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 208/413] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Sep 1 11:57:44 2022] PUT /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a/groups/d4255c975d8b42a3bcb63bbbe285a5e9/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:44.416048 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:44.432745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6191e6-e8fc-436a-9759-f56a6fb69a06 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.482140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:44.482824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:44.483108 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:44.483437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:44.484707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:44.485016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:44.486583 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:44.497095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 206/414] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:44.504126 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:44.569679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fda2c-9072-4d42-91d2-5427d10c21ac None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.571726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:44.630201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:44.631799 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:44.632919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:44.633460 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:44.636954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:44.651031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:44.656328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a853-4dac-4376-b1db-9ca07b23ea8f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:44.659228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:44.660228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:44.661014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:44.663940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 209/415] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:44.672125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:44.699915 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:44.700572 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:44.700849 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:44.701182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] PATH_INFO: `/v3/groups/083e2d9a0bc74297ad45d3e9b115cc0c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:44.702379 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] RBAC: Authorizing `identity:get_group(group_id=083e2d9a0bc74297ad45d3e9b115cc0c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:44.704301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:44.709750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 207/416] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:44 2022] GET /identity/v3/groups/083e2d9a0bc74297ad45d3e9b115cc0c => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:44.713665 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:44.857292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571adc32-ba48-4ab8-a7bb-b941497b78bf None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.898496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded723d-40f5-42f7-8c4f-ab1d57629972 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:44.911448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:44.912083 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:44.912366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:44.912697 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:44.913387 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:44.913724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:44.915109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:44.940900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 210/417] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:44.954160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:44.965255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:45.022199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:45.024081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:45.024879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:45.025704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] PATH_INFO: `/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:45.028593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] RBAC: Authorizing `identity:get_project(project_id=d6c17f90daea4fc3b1b162a692b4ec70)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:45.034416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:45.034416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 208/418] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:44 2022] GET /identity/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70 => generated 344 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:45.047254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:45.070164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:45.154166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e48a4-6540-4b0f-809f-f5d393874a5a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:45.202239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693742da-cba0-4c4e-b475-8a24076ae88c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:45.235058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:45.236613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:45.236613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:45.236613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:45.259107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0152ff-6617-49f2-82f7-ce7ee2d8a7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:45.259788 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0152ff-6617-49f2-82f7-ce7ee2d8a7ca None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:45.260117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0152ff-6617-49f2-82f7-ce7ee2d8a7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:45.260360 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0152ff-6617-49f2-82f7-ce7ee2d8a7ca None admin] PATH_INFO: `/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70/groups/083e2d9a0bc74297ad45d3e9b115cc0c/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:45.267142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0152ff-6617-49f2-82f7-ce7ee2d8a7ca None admin] RBAC: Authorizing `identity:create_grant(project_id=d6c17f90daea4fc3b1b162a692b4ec70, group_id=083e2d9a0bc74297ad45d3e9b115cc0c, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:45.268647 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0152ff-6617-49f2-82f7-ce7ee2d8a7ca None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:45.316872 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:45.318426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:45.349644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 209/419] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Sep 1 11:57:45 2022] PUT /identity/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70/groups/083e2d9a0bc74297ad45d3e9b115cc0c/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:57:45.405701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:45.449331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba720a-ee8f-4af0-ab7b-ff70e8ffd8e5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:45.449331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba720a-ee8f-4af0-ab7b-ff70e8ffd8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:45.449331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba720a-ee8f-4af0-ab7b-ff70e8ffd8e5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:45.449331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 210/420] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:45.464488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a464-701f-481a-8840-d982b35d30ce None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:45.465378 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a464-701f-481a-8840-d982b35d30ce None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:45.466178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a464-701f-481a-8840-d982b35d30ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:45.476038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 211/421] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:45.476281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:45.525645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1c95a464-701f-481a-8840-d982b35d30ce None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:45.528570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1c95a464-701f-481a-8840-d982b35d30ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:45.632933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f649c67-3c76-46d8-9ff9-b80b6587c55b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:45.675459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:45.676008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:45.676284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:45.676561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:45.677765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:45.679252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:45.682008 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:45.685725 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 212/422] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:45 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:45.687772 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:45.703218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 211/423] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:45.723765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:45.863288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c7a66-d5da-4dd3-9100-953a3d9ba5c8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:45.935518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:45.936438 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:45.936863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:45.937349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:45.938773 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:45.940408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:45.942261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 213/424] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:45 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:45.949629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a464-701f-481a-8840-d982b35d30ce None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:46.115236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a464-701f-481a-8840-d982b35d30ce None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:46.166593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:46.167446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:46.167828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:46.168206 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:46.169053 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:46.169492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:46.171093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:46.182535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 212/425] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:45 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 611 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:46.257213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:46.449989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:46.461248 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e0752-d035-40ee-b3f1-739e1cf44c7f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:46.528252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972e9b9-4b3e-4dd6-a4a4-dd01bd11ebd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:46.529212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e9b9-4b3e-4dd6-a4a4-dd01bd11ebd7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:46.529722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e9b9-4b3e-4dd6-a4a4-dd01bd11ebd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:46.530210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e9b9-4b3e-4dd6-a4a4-dd01bd11ebd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:46.550994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972e9b9-4b3e-4dd6-a4a4-dd01bd11ebd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c972e9b9-4b3e-4dd6-a4a4-dd01bd11ebd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:46.560841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:46.561847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 11:57:46.562694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:46.563893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 214/426] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:46.572667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:46.633285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:46.754271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25277d0-448a-44c2-91bc-1e57153da828 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:46.805800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5e5a-ce66-47a2-9073-c4620e44c291 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:46.806334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5e5a-ce66-47a2-9073-c4620e44c291 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:46.806577 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5e5a-ce66-47a2-9073-c4620e44c291 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:46.810754 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 215/427] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:46.820193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49da06-3af4-4510-a80b-21fefee389b2 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:46.820490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49da06-3af4-4510-a80b-21fefee389b2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:46.820840 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49da06-3af4-4510-a80b-21fefee389b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:46.824184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:46.824820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:46.825096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:46.825365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:46.826129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:46.826434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:46.828359 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:46.839307 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 213/428] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:57:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:46.848161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:46.862817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-7f49da06-3af4-4510-a80b-21fefee389b2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:46.864091 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7f49da06-3af4-4510-a80b-21fefee389b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:46.974873 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 216/429] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:46.995602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:46.999163 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8250a8-d9d6-4843-be21-089b5e566d60 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:47.044009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:47.044634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.044915 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.045184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.128013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:47.129442 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:47.248645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:47.254190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 214/430] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:57:47.262814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49da06-3af4-4510-a80b-21fefee389b2 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:47.356506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:47.369854 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:47.391272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49da06-3af4-4510-a80b-21fefee389b2 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:47.439531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6a1c-fcce-4cea-8c78-130f78a92a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:47.440556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6a1c-fcce-4cea-8c78-130f78a92a6f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.441004 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6a1c-fcce-4cea-8c78-130f78a92a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.441441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6a1c-fcce-4cea-8c78-130f78a92a6f None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.443397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6a1c-fcce-4cea-8c78-130f78a92a6f None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:47.454938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6a1c-fcce-4cea-8c78-130f78a92a6f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:47.458625 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 217/431] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:47 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:47.462541 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e46014-27cb-41d7-9693-ec247d13fd65 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.462877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e46014-27cb-41d7-9693-ec247d13fd65 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.463256 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e46014-27cb-41d7-9693-ec247d13fd65 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.465671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 218/432] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:57:47.467540 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e46014-27cb-41d7-9693-ec247d13fd65 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:47.543641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c3b17-b4d7-468d-b294-d5076f8cfb6c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:47.586796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15e4ff-2399-4770-b2a4-b01e8cc98bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:47.587438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15e4ff-2399-4770-b2a4-b01e8cc98bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.587719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15e4ff-2399-4770-b2a4-b01e8cc98bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.587994 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15e4ff-2399-4770-b2a4-b01e8cc98bc7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.601086 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15e4ff-2399-4770-b2a4-b01e8cc98bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5d15e4ff-2399-4770-b2a4-b01e8cc98bc7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:47.607027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:47.607966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:47.609393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 215/433] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:47.610672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5a81d-64d1-41fb-919d-9cfc1f93456d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.610672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5a81d-64d1-41fb-919d-9cfc1f93456d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.610672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5a81d-64d1-41fb-919d-9cfc1f93456d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.615448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e46014-27cb-41d7-9693-ec247d13fd65 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:47.632725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a2b5a81d-64d1-41fb-919d-9cfc1f93456d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:47.634000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a2b5a81d-64d1-41fb-919d-9cfc1f93456d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:47.661086 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:47.661681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.661963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.662242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.663409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:47.664864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:47.667813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 219/434] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:47 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:47.667813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:47.691443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:47.732760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:47.757762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 216/435] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:47.757762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5a81d-64d1-41fb-919d-9cfc1f93456d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:47.788705 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeac9e-cd1a-4508-9257-c88620b1caf2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:47.834789 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:47.836725 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.837167 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.837941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.838913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:47.839363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:47.841274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:47.852790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 220/436] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:47.863353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:47.924965 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5a81d-64d1-41fb-919d-9cfc1f93456d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:47.982754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:47.983994 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:47.984479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:47.985221 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:47.987358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=bddb19f21b7f472589b9154cb2aeeecc, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:47.987672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:47.990712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:48.010176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 217/437] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:47 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=bddb19f21b7f472589b9154cb2aeeecc => generated 611 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:48.012658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:48.029381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9be1c-d066-42d5-8818-8be974d58579 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:48.030095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:48.072874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff9195-c052-4f60-8d28-dc2b9fa82437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:48.073519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff9195-c052-4f60-8d28-dc2b9fa82437 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.073836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff9195-c052-4f60-8d28-dc2b9fa82437 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.074109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff9195-c052-4f60-8d28-dc2b9fa82437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.149833 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:48.160991 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff9195-c052-4f60-8d28-dc2b9fa82437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:48.162721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff9195-c052-4f60-8d28-dc2b9fa82437 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:48.222813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394ff9f-540c-49ee-898a-841610dc01ee None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:48.284272 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504b57-2887-4093-beb6-92c89a993aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:48.285828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504b57-2887-4093-beb6-92c89a993aef None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.286537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504b57-2887-4093-beb6-92c89a993aef None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.286992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504b57-2887-4093-beb6-92c89a993aef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.293357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:48.305885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a504b57-2887-4093-beb6-92c89a993aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4a504b57-2887-4093-beb6-92c89a993aef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:48.313009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:48.314103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:48.315005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 218/438] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:48.316309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bfef5-cbe0-4d6a-9def-744907a81a1b None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.316309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bfef5-cbe0-4d6a-9def-744907a81a1b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.316309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bfef5-cbe0-4d6a-9def-744907a81a1b None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.317527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 219/439] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:48.320843 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198bfef5-cbe0-4d6a-9def-744907a81a1b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:48.324394 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 221/440] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:57:48.325666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d148258-15e2-402f-9018-f55feea3d1a2 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.326526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d148258-15e2-402f-9018-f55feea3d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.326870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d148258-15e2-402f-9018-f55feea3d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.358658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-9d148258-15e2-402f-9018-f55feea3d1a2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:48.361316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9d148258-15e2-402f-9018-f55feea3d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:48.459379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 222/441] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:48.462914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198bfef5-cbe0-4d6a-9def-744907a81a1b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:48.463110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d148258-15e2-402f-9018-f55feea3d1a2 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:48.481847 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:48.517508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:48.518595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.519901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.520378 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.521588 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:48.522149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:48.524744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:48.541191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 220/442] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:48.550281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:48.633743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d148258-15e2-402f-9018-f55feea3d1a2 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:48.659127 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:48.686173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735f1e2-5740-453b-a3aa-f4ae20b9c0ee None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:48.699862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:48.700940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.701403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.701913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] PATH_INFO: `/v3/projects/8b1e5e0ec4174aac9379046a27e7794a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.704087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] RBAC: Authorizing `identity:get_project(project_id=8b1e5e0ec4174aac9379046a27e7794a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:48.706756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:48.714342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 223/443] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:48 2022] GET /identity/v3/projects/8b1e5e0ec4174aac9379046a27e7794a => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:48.716295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:48.738225 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:48.738960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.739354 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.740075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.790413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:48.820467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:48.823318 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:48.882728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff87d1-4eb9-41bb-b38b-ec9d6c2d571f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:48.943019 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:48.944150 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:48.944803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:48.945262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:48.947637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:48.950446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:48.957796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 224/444] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:48 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:48.957796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:49.011525 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 221/445] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:57:49.012638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:49.038997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:49.106783 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949385b-209e-4fee-87f8-0bb6f7e9e9b8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:49.123898 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:49.171041 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:49.172381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:49.172694 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:49.172932 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:49.173737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755d960-741b-495e-a011-d588fb2f6ef1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:49.190589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:49.197260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:49.199092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:57:49.200812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:49.202101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 225/446] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:57:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:49.203756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:49.237491 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:49.238509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:49.238963 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:49.239415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:49.240721 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=8b1e5e0ec4174aac9379046a27e7794a) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:49.241231 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:49.243949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:49.254837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 222/447] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:49 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=8b1e5e0ec4174aac9379046a27e7794a&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 611 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:49.256168 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:49.342525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b78d4-9e17-4d6a-a08f-c7544740dbc9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:49.360692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:49.403180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:49.404064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:49.404467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:49.405046 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] PATH_INFO: `/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:49.407061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] RBAC: Authorizing `identity:get_project(project_id=ba34f3047cae4d5d8da4129e3b26f8b3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:49.409120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:49.411644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 226/448] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:49 2022] GET /identity/v3/projects/ba34f3047cae4d5d8da4129e3b26f8b3 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:49.418822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:49.451063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6699a-1738-4e51-a0a3-e34d6408f1cf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:49.510300 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:49.511185 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:49.511634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:49.512072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:49.513087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:49.513518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:49.515592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:49.528087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 223/449] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:49.535299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:49.538901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:49.560623 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fb98d-f412-40e2-9de3-9675ba40cf1d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:49.599120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:49.599685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:49.599954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:49.600226 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] PATH_INFO: `/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:49.601388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] RBAC: Authorizing `identity:get_group(group_id=d4255c975d8b42a3bcb63bbbe285a5e9)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:49.602918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:49.604780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 227/450] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:49 2022] GET /identity/v3/groups/d4255c975d8b42a3bcb63bbbe285a5e9 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:49.610713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:49.670414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:49.681773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4198659-595d-456d-8819-ff22e4cf9ae5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:49.721618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:49.722360 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:49.722705 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:49.723046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:49.749969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bcf2a-220b-4cd7-b688-4e087692464b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:49.793642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:49.794384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:49.794821 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:49.795185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:49.796244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] RBAC: Adding query filter params (group.id=d4255c975d8b42a3bcb63bbbe285a5e9, role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:49.797226 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:49.798800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:49.803463 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:49.804950 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:49.806169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 228/451] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:49 2022] GET /identity/v3/role_assignments?group.id=d4255c975d8b42a3bcb63bbbe285a5e9&scope.project.id=ba34f3047cae4d5d8da4129e3b26f8b3&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:49.807189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:49.946732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 224/452] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:57:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 11:57:49.948509 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:50.082021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57686050-84bd-4f31-b72b-7a177c42cfb2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:50.100647 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:50.148212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:50.149342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:50.149859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:50.150208 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] PATH_INFO: `/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:50.152452 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] RBAC: Authorizing `identity:get_project(project_id=d6c17f90daea4fc3b1b162a692b4ec70)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:50.155316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:50.158988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 229/453] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:57:49 2022] GET /identity/v3/projects/d6c17f90daea4fc3b1b162a692b4ec70 => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:50.167132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:50.310969 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641cc31-cee2-4e41-be50-706473711a2c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:50.313761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:50.363538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a671-abf8-41e0-8af7-db834984e197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:50.364354 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a671-abf8-41e0-8af7-db834984e197 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:50.364725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a671-abf8-41e0-8af7-db834984e197 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:50.365089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a671-abf8-41e0-8af7-db834984e197 None admin] PATH_INFO: `/v3/groups/083e2d9a0bc74297ad45d3e9b115cc0c` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:50.366656 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a671-abf8-41e0-8af7-db834984e197 None admin] RBAC: Authorizing `identity:get_group(group_id=083e2d9a0bc74297ad45d3e9b115cc0c)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:50.368486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a671-abf8-41e0-8af7-db834984e197 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:50.370932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 225/454] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Sep 1 11:57:50 2022] GET /identity/v3/groups/083e2d9a0bc74297ad45d3e9b115cc0c => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:50.380882 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:50.522077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71e1cf-e278-45dd-a63d-8deb1c2f40bb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:50.577404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2a62d-6b91-4257-9379-592ab2486129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:50.578008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a62d-6b91-4257-9379-592ab2486129 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:50.578283 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a62d-6b91-4257-9379-592ab2486129 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:50.578557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2a62d-6b91-4257-9379-592ab2486129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:50.579291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a62d-6b91-4257-9379-592ab2486129 None admin] RBAC: Adding query filter params (group.id=083e2d9a0bc74297ad45d3e9b115cc0c, role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=d6c17f90daea4fc3b1b162a692b4ec70) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:50.579600 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a62d-6b91-4257-9379-592ab2486129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:50.581054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2a62d-6b91-4257-9379-592ab2486129 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:50.587670 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 230/455] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Sep 1 11:57:50 2022] GET /identity/v3/role_assignments?group.id=083e2d9a0bc74297ad45d3e9b115cc0c&scope.project.id=d6c17f90daea4fc3b1b162a692b4ec70&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:53.129484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45c695-736d-42ee-92d4-08741f58ce3a None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.130041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45c695-736d-42ee-92d4-08741f58ce3a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.130041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45c695-736d-42ee-92d4-08741f58ce3a None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.131695 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 226/456] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:53.141181 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b7296-f632-4ee5-bbb4-32d5af62ed6f None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.142171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b7296-f632-4ee5-bbb4-32d5af62ed6f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.142651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b7296-f632-4ee5-bbb4-32d5af62ed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.171765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-a07b7296-f632-4ee5-bbb4-32d5af62ed6f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:53.173357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-a07b7296-f632-4ee5-bbb4-32d5af62ed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:53.242585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:53.304944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 231/457] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:53.551166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0547d5-827f-4fae-a4ac-045d986d5c36 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.551444 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0547d5-827f-4fae-a4ac-045d986d5c36 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.551923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0547d5-827f-4fae-a4ac-045d986d5c36 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.553770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 227/458] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:53.568973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90845b3-de59-4dec-aa88-5d4aefe09aac None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.569858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90845b3-de59-4dec-aa88-5d4aefe09aac None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.570340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90845b3-de59-4dec-aa88-5d4aefe09aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.614545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-d90845b3-de59-4dec-aa88-5d4aefe09aac None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:53.616553 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d90845b3-de59-4dec-aa88-5d4aefe09aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:53.645787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d43d-63c8-4740-87be-d668af614f55 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.646068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d43d-63c8-4740-87be-d668af614f55 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.646507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d43d-63c8-4740-87be-d668af614f55 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.648230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 228/459] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:53.656890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00464b-599c-48b5-8212-cc381cc47bf9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.657149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00464b-599c-48b5-8212-cc381cc47bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.657449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00464b-599c-48b5-8212-cc381cc47bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.705918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-db00464b-599c-48b5-8212-cc381cc47bf9 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:53.707412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-db00464b-599c-48b5-8212-cc381cc47bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:53.716921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 232/460] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:53.736522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90845b3-de59-4dec-aa88-5d4aefe09aac None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:53.846719 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:53.850480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90845b3-de59-4dec-aa88-5d4aefe09aac None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:53.877731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 229/461] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:53.905101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f8a1f-052d-4aec-a355-4aa659367124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:53.906852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8a1f-052d-4aec-a355-4aa659367124 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.907341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8a1f-052d-4aec-a355-4aa659367124 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.907817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8a1f-052d-4aec-a355-4aa659367124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.920051 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f8a1f-052d-4aec-a355-4aa659367124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:53.922705 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f8a1f-052d-4aec-a355-4aa659367124 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-9a2f8a1f-052d-4aec-a355-4aa659367124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:57:53.941774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:53.942984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:57:53.943952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:57:53.943952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:53.943952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:53.946479 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 233/462] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:53.955171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00464b-599c-48b5-8212-cc381cc47bf9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:53.956494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8d7d-4a84-4347-b370-8240d8fbfe59 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.956778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8d7d-4a84-4347-b370-8240d8fbfe59 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.957063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d8d7d-4a84-4347-b370-8240d8fbfe59 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:53.959295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 234/463] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:53.972947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a0761-b022-4206-8b6f-3703932dd0c2 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:53.973552 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a0761-b022-4206-8b6f-3703932dd0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:53.974089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a0761-b022-4206-8b6f-3703932dd0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.006368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-3a9a0761-b022-4206-8b6f-3703932dd0c2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:54.012106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3a9a0761-b022-4206-8b6f-3703932dd0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:54.115080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 235/464] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 11:57:54.117087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a0761-b022-4206-8b6f-3703932dd0c2 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:54.128618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:54.132312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00464b-599c-48b5-8212-cc381cc47bf9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:54.186049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:54.187592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.188355 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.189109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.190712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:54.191551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:54.194773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:54.239788 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 230/465] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:54.255493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:54.270577 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:54.293777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a0761-b022-4206-8b6f-3703932dd0c2 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:54.367740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:54.369442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.370180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.371006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.393820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:54.397663 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:57:54.433349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:54.435359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:57:54.437269 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:57:54.437269 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:54.437269 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:54.440193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 236/466] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:54.442053 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:54.497050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca22532-d6d4-49ca-a84a-18acffcfbac3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:54.558677 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:54.559940 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.560387 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.560819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.562414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:54.564408 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:54.606051 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4328d-d969-458c-bd1d-b59fae51ba8d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:54.644316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:54.645111 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.645507 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.645927 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.648251 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:54.659995 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:54.661472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:54.666319 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 231/467] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:54 2022] POST /identity/v3/users => generated 246 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:54.670169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:57:54.680275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:54.681769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:57:54.683275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:57:54.683275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:54.683275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:54.685862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 237/468] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:54.687536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:54.698275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:54.837536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07166262-6d8d-416e-ad5d-bbc530cb3516 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:54.860445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4b50-21b2-4783-9b51-10d6dbacefd0 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:54.889841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6118a-0e7a-4cfc-bcce-8ac09b1cf31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:54.890973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6118a-0e7a-4cfc-bcce-8ac09b1cf31b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.891542 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6118a-0e7a-4cfc-bcce-8ac09b1cf31b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.891820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6118a-0e7a-4cfc-bcce-8ac09b1cf31b None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.892473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6118a-0e7a-4cfc-bcce-8ac09b1cf31b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:54.892929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6118a-0e7a-4cfc-bcce-8ac09b1cf31b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:54.895032 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6118a-0e7a-4cfc-bcce-8ac09b1cf31b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:54.904286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bc53f-e1b2-4adc-9696-812797210360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:54.904926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bc53f-e1b2-4adc-9696-812797210360 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.905205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bc53f-e1b2-4adc-9696-812797210360 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.905532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bc53f-e1b2-4adc-9696-812797210360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.910604 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 232/469] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:54.915528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc40f29-9056-4687-b1ef-db3a243a18b0 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.915528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc40f29-9056-4687-b1ef-db3a243a18b0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.915528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc40f29-9056-4687-b1ef-db3a243a18b0 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.915528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 233/470] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:54.915528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc40f29-9056-4687-b1ef-db3a243a18b0 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:54.924903 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bc53f-e1b2-4adc-9696-812797210360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:54.928269 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bc53f-e1b2-4adc-9696-812797210360 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c21bc53f-e1b2-4adc-9696-812797210360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:57:54.950179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:54.951180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:57:54.952000 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:57:54.952000 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:54.952000 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:54.952000 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 238/471] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:54.953034 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8a4f3-11b1-49b5-be21-fe3684746e89 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:54.953628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8a4f3-11b1-49b5-be21-fe3684746e89 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:54.954088 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8a4f3-11b1-49b5-be21-fe3684746e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:54.974157 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-7ef8a4f3-11b1-49b5-be21-fe3684746e89 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:54.975380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7ef8a4f3-11b1-49b5-be21-fe3684746e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:55.015800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:55.026566 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:55.041234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc40f29-9056-4687-b1ef-db3a243a18b0 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:55.080566 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 239/472] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:55.081433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8a4f3-11b1-49b5-be21-fe3684746e89 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:55.106463 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:55.107844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:55.108348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:55.109686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:55.111231 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:55.112038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:55.114482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:55.138036 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 234/473] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:55.138036 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:55.218841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8a4f3-11b1-49b5-be21-fe3684746e89 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:55.266232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:55.267227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:55.267708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:55.268179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:55.269301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:55.271090 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:55.272191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7584a1e-fd0f-4589-bf5b-53e9da4d7c8f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:55.310737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:55.311357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:55.311672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:55.312418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:55.313236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:55.313540 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:55.314952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:55.318282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:55.333141 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 235/474] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:55.337793 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:55.369796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 240/475] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:55 2022] POST /identity/v3/users => generated 248 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:55.369796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:55.470462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:55.587576 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc42d07-c6f8-4834-b9c0-6231a4e718de None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:55.594710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92818929-ceb6-4f02-9b49-c321fe7fef10 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:55.637028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:55.637562 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:55.637900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:55.638213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:55.638832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:55.640825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:55.662294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:55.669216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:55.669216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:55.669216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:55.669216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:55.669216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:55.751815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 241/476] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:55 2022] POST /identity/v3/users => generated 249 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:55.753717 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:55.813263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 236/477] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:57:55 2022] POST /identity/v3/users => generated 248 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:57:55.917520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:55.971763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760790cc-2f79-4fa2-82e0-4ea78a46fc36 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:56.048722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:56.050145 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:56.050791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:56.051417 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:56.052674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:56.055244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:56.108606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.sql.core [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 11:57:56.108606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 11:57:56.108606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [parameters: {'id': '45cd2bbb9e234ca88ba094edce108926', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 11:57:56.108606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81861) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application conn.query(q) Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result.read() Sep 01 11:57:56.131718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 11:57:56.133461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application next(self.gen) Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application yield resource Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application next(self.gen) Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise self.value Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application session.commit() Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 11:57:56.135992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.session.flush() Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application compat.raise_( Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 11:57:56.137803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 11:57:56.140457 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application conn.query(q) Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result.read() Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 11:57:56.146931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [parameters: {'id': '45cd2bbb9e234ca88ba094edce108926', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:56.150125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 11:57:56.153182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:56.158536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 242/478] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Sep 1 11:57:55 2022] POST /identity/v3/roles => generated 144 bytes in 381 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:56.158763 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:56.241291 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:56.338386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b64fa6-405f-409a-bfd9-8ae39a988bd4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:56.411622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ddc8d-3f9f-401b-bb9d-448c91e540b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:56.412693 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ddc8d-3f9f-401b-bb9d-448c91e540b7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:56.413188 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ddc8d-3f9f-401b-bb9d-448c91e540b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:56.413685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ddc8d-3f9f-401b-bb9d-448c91e540b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:56.443299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ddc8d-3f9f-401b-bb9d-448c91e540b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-178ddc8d-3f9f-401b-bb9d-448c91e540b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:56.458150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:56.460345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:56.462386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 237/479] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:57:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:56.466749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:56.673836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a07f2-086e-493e-9569-b058e2a0359f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:56.742762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d000646-306a-4395-a581-1bafb9a246f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:56.743819 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d000646-306a-4395-a581-1bafb9a246f9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:56.744319 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d000646-306a-4395-a581-1bafb9a246f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:56.744805 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d000646-306a-4395-a581-1bafb9a246f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:56.746067 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d000646-306a-4395-a581-1bafb9a246f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:56.746596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d000646-306a-4395-a581-1bafb9a246f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:56.749153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d000646-306a-4395-a581-1bafb9a246f9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:56.765437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 243/480] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:57:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:57:57.486595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c714dd-fe31-4916-96b3-87f803b04ca1 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:57.487150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c714dd-fe31-4916-96b3-87f803b04ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:57.487552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c714dd-fe31-4916-96b3-87f803b04ca1 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:57.490427 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 238/481] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:57.499109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf3c7e-147d-4d9f-a2a1-be890d68689a None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:57.500268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf3c7e-147d-4d9f-a2a1-be890d68689a None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:57.500750 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf3c7e-147d-4d9f-a2a1-be890d68689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:57.524742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:57.536109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-4fdf3c7e-147d-4d9f-a2a1-be890d68689a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:57.537327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-4fdf3c7e-147d-4d9f-a2a1-be890d68689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:57.638407 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 244/482] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:57.868595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70635db-7cc1-4f78-b30b-6df14280860c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:57.869097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70635db-7cc1-4f78-b30b-6df14280860c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:57.869549 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70635db-7cc1-4f78-b30b-6df14280860c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:57.872528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 239/483] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:57.880628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421ce56-6dd5-4514-897c-e3426ee5ca3e None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:57.881013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421ce56-6dd5-4514-897c-e3426ee5ca3e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:57.881188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421ce56-6dd5-4514-897c-e3426ee5ca3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:57.913414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-d421ce56-6dd5-4514-897c-e3426ee5ca3e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:57.914698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d421ce56-6dd5-4514-897c-e3426ee5ca3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:58.023154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 245/484] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:57:58.313783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70635db-7cc1-4f78-b30b-6df14280860c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:58.471499 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70635db-7cc1-4f78-b30b-6df14280860c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:58.490488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:58.530318 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:58.531009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:58.531287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:58.531554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:58.554609 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:58.559661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:58.561365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:58.563622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 240/485] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:57:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:58.574882 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421ce56-6dd5-4514-897c-e3426ee5ca3e None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:58.672521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:58.702198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:58.797712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421ce56-6dd5-4514-897c-e3426ee5ca3e None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:58.872629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:58.874291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:58.875055 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:58.875806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:58.877344 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:58.878397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:58.881924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:58.903011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 246/486] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:57:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:58.913384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:58.947508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c62d-70bf-44b8-918a-c62e6bea9beb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:58.991870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948f099-4137-4aca-b49a-c9e02bd83181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:58.992805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948f099-4137-4aca-b49a-c9e02bd83181 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:58.993086 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948f099-4137-4aca-b49a-c9e02bd83181 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:58.993639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948f099-4137-4aca-b49a-c9e02bd83181 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.010495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948f099-4137-4aca-b49a-c9e02bd83181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b948f099-4137-4aca-b49a-c9e02bd83181 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:59.017013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:59.018784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 241/487] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:57:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:57:59.020305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b1751-52fb-448f-9936-2135eedcfe50 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.022654 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b1751-52fb-448f-9936-2135eedcfe50 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.022654 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b1751-52fb-448f-9936-2135eedcfe50 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.023402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 242/488] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:59.025719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef18909-fe2d-4d1e-a1a3-f31065cce9c8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.026212 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef18909-fe2d-4d1e-a1a3-f31065cce9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.026706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef18909-fe2d-4d1e-a1a3-f31065cce9c8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.030322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 243/489] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:59.031029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef18909-fe2d-4d1e-a1a3-f31065cce9c8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:59.053693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49154f55-292f-4aae-91ef-495d608591c8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:59.082737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:59.087944 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:59.091617 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecde57-acbc-4d62-986f-3228b1968c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:59.095939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecde57-acbc-4d62-986f-3228b1968c86 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.095939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecde57-acbc-4d62-986f-3228b1968c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.095939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecde57-acbc-4d62-986f-3228b1968c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.105012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecde57-acbc-4d62-986f-3228b1968c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:59.106535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecde57-acbc-4d62-986f-3228b1968c86 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-d0ecde57-acbc-4d62-986f-3228b1968c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:57:59.122601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:59.123634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:57:59.125128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:57:59.125128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:59.125128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:59.125128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 247/490] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:59.125900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2531-49b4-4124-8259-85b70d8d0eff None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.126012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2531-49b4-4124-8259-85b70d8d0eff None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.126340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2531-49b4-4124-8259-85b70d8d0eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.146609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-786f2531-49b4-4124-8259-85b70d8d0eff None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:59.148061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-786f2531-49b4-4124-8259-85b70d8d0eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:59.195723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef18909-fe2d-4d1e-a1a3-f31065cce9c8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:59.235899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 248/491] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:59.235899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09346a-7ebc-480e-9d01-3a4ad83d25e3 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.235899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09346a-7ebc-480e-9d01-3a4ad83d25e3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.235899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09346a-7ebc-480e-9d01-3a4ad83d25e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.255371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-0c09346a-7ebc-480e-9d01-3a4ad83d25e3 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:57:59.256604 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0c09346a-7ebc-480e-9d01-3a4ad83d25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:57:59.273837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:59.276067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.276580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.277431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.279207 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:59.280210 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:59.282884 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:59.307546 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 244/492] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:57:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:59.309797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:59.379686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 249/493] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:57:59.379686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09346a-7ebc-480e-9d01-3a4ad83d25e3 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:59.500279 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:59.603275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09346a-7ebc-480e-9d01-3a4ad83d25e3 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:59.619602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6ab41-17ec-4e38-be88-e22a46e1606b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:59.625918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:59.644447 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:59.645006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.645361 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.645707 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.659436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:59.660989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:57:59.678354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:59.680373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:57:59.682388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:57:59.682388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:57:59.682388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:59.684403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 250/494] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:59.685378 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:59.694111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:59.695472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.696043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.696628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.697959 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:59.698673 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:59.701446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:59.725415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 245/495] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:59.727057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:59.804909 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:57:59.814107 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece8ba5-625f-4330-997b-1455d7d7f734 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:59.869936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:59.871221 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.871803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.872258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.888282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe41256-f4d0-4bb2-a3de-6c8dd9c85e80 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:57:59.906206 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:57:59.916139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:57:59.917147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:57:59.917997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:57:59.918603 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 251/496] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:57:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:57:59.919924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:59.936111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270bc0b-90a6-455a-b5e2-2f2e76cf8ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:57:59.937218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270bc0b-90a6-455a-b5e2-2f2e76cf8ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.937501 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270bc0b-90a6-455a-b5e2-2f2e76cf8ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.937870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270bc0b-90a6-455a-b5e2-2f2e76cf8ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.938529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270bc0b-90a6-455a-b5e2-2f2e76cf8ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:57:59.938840 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270bc0b-90a6-455a-b5e2-2f2e76cf8ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:57:59.940991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270bc0b-90a6-455a-b5e2-2f2e76cf8ef6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:57:59.956555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 246/497] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:57:59.957812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2d2b0-bef8-4636-a8bd-1800ca06aa5a None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:57:59.958113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2d2b0-bef8-4636-a8bd-1800ca06aa5a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:57:59.958390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2d2b0-bef8-4636-a8bd-1800ca06aa5a None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:57:59.959727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 247/498] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:57:59.961651 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2d2b0-bef8-4636-a8bd-1800ca06aa5a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:57:59.984727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:00.061395 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c77ce-b19c-460b-8597-d7db87b9a0d5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.095541 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2d2b0-bef8-4636-a8bd-1800ca06aa5a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.101154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.101847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.102136 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.102413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.111528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:00.113020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:00.127853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:00.129650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 252/499] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:57:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:00.131396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.142025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.142938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.143345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.143681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.144406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:00.144767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:00.146582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:00.149342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:00.163787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 248/500] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:57:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:00.163787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.241423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:00.248667 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c32c6c-764e-4f9a-8879-c4ee6ed0e525 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.264382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1692da-a3d5-4b71-89f3-4ca920aaf970 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.287019 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cb757-bc32-4464-a7b7-11f2cb2ccdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.287696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cb757-bc32-4464-a7b7-11f2cb2ccdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.287948 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cb757-bc32-4464-a7b7-11f2cb2ccdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.288215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cb757-bc32-4464-a7b7-11f2cb2ccdc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.299959 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639cb757-bc32-4464-a7b7-11f2cb2ccdc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:00.303263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.303894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.304171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.304437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-639cb757-bc32-4464-a7b7-11f2cb2ccdc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:00.304973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:00.306557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 253/501] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:00.308077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889064d6-7936-4eed-ad53-8faf4e768112 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.309525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889064d6-7936-4eed-ad53-8faf4e768112 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.309525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889064d6-7936-4eed-ad53-8faf4e768112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.319117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:00.320848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:00.328320 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-889064d6-7936-4eed-ad53-8faf4e768112 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:00.329705 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-889064d6-7936-4eed-ad53-8faf4e768112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:00.335209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:00.341754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 249/502] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:00.342646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.403614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 254/503] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:00.404776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889064d6-7936-4eed-ad53-8faf4e768112 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.408887 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:00.457266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cf759-33d5-4644-9344-ac5070eef07b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.497839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.498732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.499012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.499800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.500536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:00.500842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:00.502326 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:00.524652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:00.545016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889064d6-7936-4eed-ad53-8faf4e768112 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.558008 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 250/504] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:00.558751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.592699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.594015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.594312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.594969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.607697 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:00.610034 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:00.626642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:00.628437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:00.630176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:00.630176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:00.630176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:00.630176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 255/505] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:00.630176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.693074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb04b6-f196-4c95-bfbb-2016b85f4af8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.699262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:00.747470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79948207-8176-4830-b94a-e7a2ccfaf316 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.756144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.757242 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.757750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.758239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.759438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:00.759979 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:00.762440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:00.781877 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 251/506] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:00.781877 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.782933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120791dc-d728-466b-a3be-048890723d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.783967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120791dc-d728-466b-a3be-048890723d73 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.784245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120791dc-d728-466b-a3be-048890723d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.784521 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120791dc-d728-466b-a3be-048890723d73 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.785263 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120791dc-d728-466b-a3be-048890723d73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:00.785569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120791dc-d728-466b-a3be-048890723d73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:00.787045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120791dc-d728-466b-a3be-048890723d73 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:00.827058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 256/507] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:00.828435 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120791dc-d728-466b-a3be-048890723d73 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:00.867966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:00.913184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3aa39-0f70-48f6-9db3-6d4d32b5610b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.947078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:00.947692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:00.947970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:00.948240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:00.979381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120791dc-d728-466b-a3be-048890723d73 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:00.994518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.014914 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.015419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.015696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.015990 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.016634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:01.016932 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.018352 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.028895 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.030393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.030883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 257/508] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:01.032235 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.099413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.144611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bfed-31e8-4e72-879a-1807620f1ad2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.160422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 252/509] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:01.160422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.193043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.193718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.194003 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.194526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.206319 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.207795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:01.223045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:01.224653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:01.225778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:01.225778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:01.225778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:01.225778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 258/510] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:01.225778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.256245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.280601 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ddca8-9e62-426b-bb34-239d9979dffb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.313133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.313816 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.314096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.314371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.325441 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.326953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.335199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4e4e5-2f74-4d6f-8796-16592e7b0fbf None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:01.347777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:01.348823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:01.350162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:01.350162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:01.350162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:01.350162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 253/511] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:01.350162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.363404 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.381862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.382479 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.382762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.383038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.458765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.460261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.471824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da050-bef4-4833-be01-0a0815426a21 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.485879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.507810 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.508512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.508795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.509070 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.517970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.519544 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:01.532887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:01.540400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 254/512] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:01.542346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.570000 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 259/513] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:01.570941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.610350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.647774 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a6b9-a456-405e-a87b-26b8474b0501 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.683392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3874b-3746-4ba6-86f9-e50b11de4edb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.685736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.686379 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.686655 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.687449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.698889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.700358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.709065 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:01.722120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:01.723118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:01.723960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:01.723960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:01.723960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:01.724132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 255/514] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:01.725140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.729621 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.730591 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.731031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.731468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.732841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:01.733342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.735260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.759498 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 260/515] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:01.759498 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.872398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85265ea-6129-4d9c-bbd9-1ab2562928e8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.889591 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d449fa4-5f06-41e5-a14c-e8684c58323c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:01.896609 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:01.913754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.914731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.915185 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.915633 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.916516 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:01.917012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.919466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.933190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:01.933844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:01.934190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:01.934478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:01.934621 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 256/516] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:01.935219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:01.935376 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:01.935499 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:01.937463 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:01.963925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 261/517] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:01.964856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.013437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:02.055367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ea2ac-b430-4eb0-a338-cdff7cee9cce None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.072640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32c7cc-e59e-40f8-a9a8-aebc618c5d65 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.095939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.096667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.097088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.097456 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.108712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.109711 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.110235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:02.110365 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.110592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.110860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.111513 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:02.111809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.113178 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:02.126947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:02.128515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:02.129459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:02.129459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:02.129459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:02.129459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 257/518] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:02.129459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.130968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 262/519] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:02.132779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.165270 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:02.245771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31164837-9120-4ec6-b023-98c7abab1170 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.250140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495af51d-c882-4238-9a0f-6fb1473131c6 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.251942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:02.284455 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.284766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.284899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.285127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.285820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.287294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:02.297026 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.298292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.298817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.299201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.308547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.310130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:02.322481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:02.330303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 263/520] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:02.331232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.395060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 258/521] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:58:02 2022] POST /identity/v3/users => generated 247 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:02.395838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.469069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f12f63-a01e-4161-86dd-195b1ffbdc93 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.521225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.521911 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.522675 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.522675 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.536894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:02.552378 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7316-e355-4905-a665-3be4a7f31127 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.563689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:02.610994 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.611662 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.611994 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.612292 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.620907 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.622074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.622830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:02.625381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:02.640506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:02.641464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:02.642791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 259/522] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:02.643256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.765973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a27a5-dd1b-4e4a-b915-d567f3674b46 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.800792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 264/523] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:02.800792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.834280 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.835185 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.835465 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.835738 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.836596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:02.836904 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.838766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:02.863304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 260/524] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:02.863304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:02.879402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:02.894473 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:02.946294 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70c0b-a6d9-4630-8df1-40d2c5daa7b9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:02.986223 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:02.986984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:02.987508 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:02.987995 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:02.988860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:02.989239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:02.990823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.013332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 265/525] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Sep 1 11:58:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:03.014244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:03.051708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424e674-bc7b-43e9-99df-75a6dc44336a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:03.096217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:03.097317 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:03.097594 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:03.097896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:03.098707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:03.098890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:03.100375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.120548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 261/526] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:03.121412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:03.198690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a016bf-986f-47d0-9dec-9336139101d4 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:03.229962 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:03.238906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:03.262103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:03.262827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:03.263110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:03.263385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:03.270942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b4e4-66ac-4cff-ab21-6d7e60fada0b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:03.283709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:03.286492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:03.315326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:03.317305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:03.319054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:03.319054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:03.319054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:03.322612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 266/527] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:03.324310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:03.325640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:03.326313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:03.326599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:03.326864 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:03.338325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:03.340108 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:03.357050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:03.359078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 262/528] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:03.360686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:03.489229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089403af-48b5-442c-a266-e124f2094da5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:03.492704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a7862-f2b9-4dba-b8d3-6fa1cb0dadaa None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:03.527102 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:03.534480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:03.535382 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:03.535880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:03.536372 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:03.536679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:03.537391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:03.537699 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:03.537972 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:03.538706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=0c8225879a66415c8a1dc28584e4c626) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:03.539012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:03.540452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.548045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 263/529] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/role_assignments?user.id=0c8225879a66415c8a1dc28584e4c626&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:03.549200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:03.624113 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:03.660686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:03.662283 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.745041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18f548-4559-45a4-be6b-8a7b2e3f5710 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:03.786414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 267/530] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:03.787486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:03.806944 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:03.807938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:03.808434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:03.808919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:03.810208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:03.810755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:03.813971 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.838628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 264/531] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:03.840275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:03.867905 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:03.901947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe61584-727a-42aa-bc26-425bf22f34d6 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:03.941297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:03.942019 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:03.942377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:03.942798 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:03.943247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:03.944090 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:03.944787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:03.946417 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:03.967175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 268/532] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:03.968366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:04.040724 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1086-fb34-4752-9c92-f3a63a22c911 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:04.081531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a2afb-c895-48bc-9759-d9e48393ccf8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:04.104875 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:04.106225 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.106471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.106967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.117582 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:04.123618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:04.125223 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.125690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.126103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.127909 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:04.130773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:04.134658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:04.137458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:04.149212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:04.150355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:04.151399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:04.153397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 269/533] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:04.155989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:04.164507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:04.165561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:04.166737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:04.166737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:04.166737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:04.166737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 265/534] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:04.166967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:04.289641 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63bed0-2d7d-48da-bc7d-e9af691d9f9f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:04.318539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b582ac-6c2a-4e07-bf24-01b7f0dcfab6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:04.323498 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:04.328645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:04.329267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.329547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.329865 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.330611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=bb79770a161740128e5792a70cbad00a) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:04.330919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:04.334860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:04.349644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 270/535] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:04 2022] GET /identity/v3/role_assignments?user.id=bb79770a161740128e5792a70cbad00a&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:04.349644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:04.408157 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:04.442471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:04.443410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.443903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.444613 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.445516 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:04.446058 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:04.447728 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:04.462682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7a1ba-0368-41e1-b36c-63a8113e76a1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:04.478265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 266/536] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:04.486313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:04.543442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232922a-edd0-4e5d-9e58-9616665060e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:04.544080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232922a-edd0-4e5d-9e58-9616665060e5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.544361 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232922a-edd0-4e5d-9e58-9616665060e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.544639 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232922a-edd0-4e5d-9e58-9616665060e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.545308 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232922a-edd0-4e5d-9e58-9616665060e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:04.545625 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232922a-edd0-4e5d-9e58-9616665060e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:04.547712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232922a-edd0-4e5d-9e58-9616665060e5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:04.568642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 271/537] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:04.570627 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eaa35-abe1-4279-978c-afe5fcd76fa0 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.570916 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eaa35-abe1-4279-978c-afe5fcd76fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.571193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eaa35-abe1-4279-978c-afe5fcd76fa0 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.572862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 272/538] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:04.581548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eaa35-abe1-4279-978c-afe5fcd76fa0 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:04.637278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ef7c2-e87c-4d90-bf15-941926e5d3ca None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:04.690689 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984ff49-d41f-4792-a8ef-c2a84a2ebedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:04.691807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984ff49-d41f-4792-a8ef-c2a84a2ebedd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.692248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984ff49-d41f-4792-a8ef-c2a84a2ebedd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.692930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984ff49-d41f-4792-a8ef-c2a84a2ebedd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.694759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984ff49-d41f-4792-a8ef-c2a84a2ebedd None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=86e8ec32ae1f46e9a1a3ada4fc7201c3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:04.695248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984ff49-d41f-4792-a8ef-c2a84a2ebedd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:04.697096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984ff49-d41f-4792-a8ef-c2a84a2ebedd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:04.697998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eaa35-abe1-4279-978c-afe5fcd76fa0 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:04.709643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 267/539] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:04 2022] GET /identity/v3/role_assignments?user.id=86e8ec32ae1f46e9a1a3ada4fc7201c3&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:04.709643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92463aa6-6850-46aa-8e92-ac76de68ce75 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.709643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92463aa6-6850-46aa-8e92-ac76de68ce75 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.709643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92463aa6-6850-46aa-8e92-ac76de68ce75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.732537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-92463aa6-6850-46aa-8e92-ac76de68ce75 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:04.737986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:04.738142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-92463aa6-6850-46aa-8e92-ac76de68ce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:04.767100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:04.767768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:04.769164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:04.769476 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:04.781346 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:04.784138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:04.790839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:04.801536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:04.802482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:04.803330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:04.804268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 273/540] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:04.810563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:04.901952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 268/541] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:05.016266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f0b53-8e01-4a32-8f51-273312e8772e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:05.066336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:05.066965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:05.067250 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:05.067525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:05.068228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:05.068534 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:05.070257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:05.091567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 274/542] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:05.097643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92463aa6-6850-46aa-8e92-ac76de68ce75 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:05.206834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:05.270078 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:05.297223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92463aa6-6850-46aa-8e92-ac76de68ce75 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:05.344964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:05.345623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:05.348204 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:05.348542 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:05.349307 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=75e0ceaa2c6745dc89ade0c43a17572b) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:05.349631 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:05.351105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:05.366070 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 269/543] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:05 2022] GET /identity/v3/role_assignments?user.id=75e0ceaa2c6745dc89ade0c43a17572b&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:05.400570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b901618-aac0-4125-89c0-590f17cbc137 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:05.441955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:05.442604 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:05.442888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:05.443160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:05.465737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:05.473587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:05.474612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:05.475408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:05.480195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 275/544] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:05.486586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:05.532519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:05.646775 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560e586-bd22-41c5-a270-b55f63f32d9c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:05.693592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:05.694600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:05.694887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:05.695351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:05.696094 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:05.696407 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:05.698276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:05.708536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 270/545] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:05.714289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:05.864371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:05.871377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e8985-80a5-4730-b229-08c519d96420 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:05.929417 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e570e64-efcc-48b5-80b1-a923b60efed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:05.930102 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570e64-efcc-48b5-80b1-a923b60efed8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:05.930387 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570e64-efcc-48b5-80b1-a923b60efed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:05.930667 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570e64-efcc-48b5-80b1-a923b60efed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:05.943139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e570e64-efcc-48b5-80b1-a923b60efed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:05.944896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e570e64-efcc-48b5-80b1-a923b60efed8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-5e570e64-efcc-48b5-80b1-a923b60efed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:05.974525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:05.975505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:05.976972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:05.976972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:05.976972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:05.976972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 276/546] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:05.984122 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:06.024411 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f46e5-2ac9-4545-8d34-b0913c9d447b None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:06.024929 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f46e5-2ac9-4545-8d34-b0913c9d447b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:06.025005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f46e5-2ac9-4545-8d34-b0913c9d447b None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:06.026637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 277/547] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:06.038758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c51bd-a51f-45cb-9414-b86faef91ab7 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:06.039080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c51bd-a51f-45cb-9414-b86faef91ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:06.039316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c51bd-a51f-45cb-9414-b86faef91ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:06.064141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-070c51bd-a51f-45cb-9414-b86faef91ab7 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:06.066902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-070c51bd-a51f-45cb-9414-b86faef91ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:06.137663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:06.171753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e988411-ffca-42c2-9935-b86cc80488fa None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:06.205356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 278/548] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 11:58:06.253626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:06.254659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:06.255147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:06.255635 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:06.256522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:06.257034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:06.258815 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:06.284082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 271/549] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:06.290385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c51bd-a51f-45cb-9414-b86faef91ab7 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:06.429564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:06.458899 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c51bd-a51f-45cb-9414-b86faef91ab7 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:06.521538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:06.522660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:06.523437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:06.523980 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:06.536990 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:06.540134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:06.563370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:06.564465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:06.565537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:06.566101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 279/550] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:06.572241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:06.586880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:06.743593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657851a-1366-4efd-918a-c2eca7fbc60d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:06.746818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04158af0-6595-40b7-8636-b3c9c18c87e7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:06.820230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:06.820880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:06.821163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:06.821435 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:06.822971 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:06.824405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:06.825203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:06.826014 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:06.827509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:06.828366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:06.831440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:06.837833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:06.851252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:06.856841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:06.859516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:06.861288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:06.868664 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 280/551] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:06.878904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:06.881557 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 272/552] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Sep 1 11:58:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:06.890270 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:06.905064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:07.067426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250d57-ef7b-4c7e-98b2-a6c4126bf169 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:07.131553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d18cb-d13b-4f04-a0cb-2a11b71392cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:07.132576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d18cb-d13b-4f04-a0cb-2a11b71392cc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.133073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d18cb-d13b-4f04-a0cb-2a11b71392cc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.133570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879041a8-1d34-4150-b1de-0e63bd06e67a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:07.134202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d18cb-d13b-4f04-a0cb-2a11b71392cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.203936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5c10c-f738-42b7-ba95-f2faa4c1ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:07.205008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5c10c-f738-42b7-ba95-f2faa4c1ce3d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.205503 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5c10c-f738-42b7-ba95-f2faa4c1ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.206062 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5c10c-f738-42b7-ba95-f2faa4c1ce3d None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.207289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5c10c-f738-42b7-ba95-f2faa4c1ce3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:07.207837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5c10c-f738-42b7-ba95-f2faa4c1ce3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:07.210616 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5c10c-f738-42b7-ba95-f2faa4c1ce3d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:07.221598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d18cb-d13b-4f04-a0cb-2a11b71392cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:07.223582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d18cb-d13b-4f04-a0cb-2a11b71392cc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:07.226122 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:07.232274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 281/553] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:07.232274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1bc34-d2bd-42c4-b526-b47da78a8dac None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.232274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1bc34-d2bd-42c4-b526-b47da78a8dac None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.232274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1bc34-d2bd-42c4-b526-b47da78a8dac None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.234094 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 282/554] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:07.235148 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1bc34-d2bd-42c4-b526-b47da78a8dac None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:07.356758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:07.363208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 273/555] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:07.365075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58114669-cdfa-45dc-bf4f-a7c6f2ba2e51 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.365727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58114669-cdfa-45dc-bf4f-a7c6f2ba2e51 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.366334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58114669-cdfa-45dc-bf4f-a7c6f2ba2e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.387255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-58114669-cdfa-45dc-bf4f-a7c6f2ba2e51 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:07.388912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-58114669-cdfa-45dc-bf4f-a7c6f2ba2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:07.447474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1bc34-d2bd-42c4-b526-b47da78a8dac None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:07.473813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 274/556] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:07.475176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58114669-cdfa-45dc-bf4f-a7c6f2ba2e51 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:07.510548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:07.510970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:07.512190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.512600 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.513141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.536202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:07.539092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:07.568759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:07.570115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:07.571010 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:07.571010 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:07.571010 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:07.571605 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 283/557] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:07.580175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:07.598311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58114669-cdfa-45dc-bf4f-a7c6f2ba2e51 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:07.640187 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876c1b1-40c6-4642-9886-1741b7f01496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:07.641234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c1b1-40c6-4642-9886-1741b7f01496 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.641783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c1b1-40c6-4642-9886-1741b7f01496 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.642300 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c1b1-40c6-4642-9886-1741b7f01496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.654716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876c1b1-40c6-4642-9886-1741b7f01496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:07.657206 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876c1b1-40c6-4642-9886-1741b7f01496 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0876c1b1-40c6-4642-9886-1741b7f01496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:07.675301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:07.676848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:07.677795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:07.677795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:07.677795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:07.680053 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 275/558] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:07.686295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28938c-e1e2-43d9-92db-96e58ed1b2c6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.687582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28938c-e1e2-43d9-92db-96e58ed1b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.688100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28938c-e1e2-43d9-92db-96e58ed1b2c6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.690184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 276/559] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:07.691277 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28938c-e1e2-43d9-92db-96e58ed1b2c6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:07.793296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe328b0-090f-4c91-a122-7b197f63154b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:07.800625 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:07.821649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28938c-e1e2-43d9-92db-96e58ed1b2c6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:07.823195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:07.865561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a28d9f-e5e1-4726-b162-0125ed43c4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:07.866731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a28d9f-e5e1-4726-b162-0125ed43c4ba None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.867324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a28d9f-e5e1-4726-b162-0125ed43c4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.867807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a28d9f-e5e1-4726-b162-0125ed43c4ba None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.868761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a28d9f-e5e1-4726-b162-0125ed43c4ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:07.869290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a28d9f-e5e1-4726-b162-0125ed43c4ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:07.870482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:07.871299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a28d9f-e5e1-4726-b162-0125ed43c4ba None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:07.871565 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.872057 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.872541 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.873782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:07.875032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:07.877758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:07.892320 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 277/560] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:07.894180 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1dbf-5a74-410a-bf76-f5785224e30c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:07.894706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1dbf-5a74-410a-bf76-f5785224e30c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:07.895217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1dbf-5a74-410a-bf76-f5785224e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:07.904791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 284/561] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:07.907108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:07.916523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-903b1dbf-5a74-410a-bf76-f5785224e30c None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:07.918129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-903b1dbf-5a74-410a-bf76-f5785224e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:07.995295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 278/562] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:07.996504 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1dbf-5a74-410a-bf76-f5785224e30c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.095665 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:08.131479 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8fc88-e2c6-4539-94da-0ce79cc35f9f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:08.143032 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1dbf-5a74-410a-bf76-f5785224e30c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:08.175947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:08.177752 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.178266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.178749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.188898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:08.191472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:08.207826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:08.208817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:08.209776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:08.216873 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 285/563] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:08.216873 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.217149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:08.217149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.217149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.217149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.232190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:08.234995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:08.258316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:08.259288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:08.260264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:08.261253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:08.266047 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 279/564] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:58:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:08.267643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.370957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c17a-33cc-497f-819e-b22ad376fc0c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:08.409518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:08.410110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.410403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.410680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.425951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:08.426750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2776f7-67fe-4a1f-b723-e6d45efaaed3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:08.435679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:08.437297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:08.438211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 286/565] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:08.439925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.472453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3915a-e041-4160-bdd2-305d3af08563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:08.473783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3915a-e041-4160-bdd2-305d3af08563 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.474147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3915a-e041-4160-bdd2-305d3af08563 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.474336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3915a-e041-4160-bdd2-305d3af08563 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.475048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3915a-e041-4160-bdd2-305d3af08563 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:08.475360 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3915a-e041-4160-bdd2-305d3af08563 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:08.477421 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3915a-e041-4160-bdd2-305d3af08563 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:08.479374 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:08.494323 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 280/566] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:08.497047 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc9ac-5096-4a0e-964b-814d60325b7b None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.497870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc9ac-5096-4a0e-964b-814d60325b7b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.498630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc9ac-5096-4a0e-964b-814d60325b7b None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.502297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 281/567] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:08.504105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc9ac-5096-4a0e-964b-814d60325b7b None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.574260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:08.588682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42cb54-2ba7-49d1-958f-40dc1c011e49 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:08.633970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc9ac-5096-4a0e-964b-814d60325b7b None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:08.641284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:08.642548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.642937 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.643298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.644540 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:08.644938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:08.646855 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:08.671770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 287/568] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:58:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:08.673084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.677624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197c08d-c011-442f-bf09-6c68522b8010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:08.678368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c08d-c011-442f-bf09-6c68522b8010 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.678748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c08d-c011-442f-bf09-6c68522b8010 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.679150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c08d-c011-442f-bf09-6c68522b8010 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.679944 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c08d-c011-442f-bf09-6c68522b8010 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:08.680414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c08d-c011-442f-bf09-6c68522b8010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:08.681929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c08d-c011-442f-bf09-6c68522b8010 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:08.691281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 282/569] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:08.692640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9087-53ce-4e89-ba02-ba33ce776733 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.693031 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9087-53ce-4e89-ba02-ba33ce776733 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.693398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a9087-53ce-4e89-ba02-ba33ce776733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.712281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b49a9087-53ce-4e89-ba02-ba33ce776733 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:08.713511 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b49a9087-53ce-4e89-ba02-ba33ce776733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:08.802702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fca42b-f096-44ea-b11e-e12291c4ed06 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:08.849841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:08.869817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:08.870586 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:08.870957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:08.871325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:08.872216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=60c4eccc7fb0434e9251fa1b9bd7aec2) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:08.872620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:08.873074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 283/570] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:08.874155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9087-53ce-4e89-ba02-ba33ce776733 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.875299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:08.890071 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 288/571] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:08 2022] GET /identity/v3/role_assignments?user.id=60c4eccc7fb0434e9251fa1b9bd7aec2&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:08.891472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:08.924665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:09.039478 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a9087-53ce-4e89-ba02-ba33ce776733 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.094531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.095610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.096093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.097032 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.097531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cf1b5-db58-4aa9-9714-4a95201576e2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.111688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:09.118137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:09.119160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:09.120024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:09.122960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 284/572] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:09.124047 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:09.167814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91532-af37-4654-989e-911b8140f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.168463 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91532-af37-4654-989e-911b8140f865 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.168737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91532-af37-4654-989e-911b8140f865 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.169106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91532-af37-4654-989e-911b8140f865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.184989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce91532-af37-4654-989e-911b8140f865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:09.186385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce91532-af37-4654-989e-911b8140f865 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-bce91532-af37-4654-989e-911b8140f865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:09.206437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:09.209085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:09.211009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:09.211009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:09.211009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:09.211009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 289/573] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:09.211009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91532-af37-4654-989e-911b8140f865 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:09.220393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:09.325438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1da80-ba13-4de9-b571-4ba66de0c2b6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.327668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:09.378720 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91532-af37-4654-989e-911b8140f865 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.406235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.407344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.407859 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.408351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.418513 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.419281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.419436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.419710 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.420523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:09.420845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:09.423011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:09.431140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:09.432503 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 290/574] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:09.433357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:09.435405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:09.471652 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:09.472704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:09.473657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:09.473657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:09.473657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:09.473657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 285/575] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:09.474783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:09.614391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:09.619441 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d54b2-8688-40e1-9856-abd254c65af2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.654654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62bb9c-120a-484e-aa2e-29f7d31eb79d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.671039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.672107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.672591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.673062 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.688805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:09.696348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:09.698472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 286/576] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:09.700113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:09.703773 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:09.718866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.720103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.720744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.721179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.722545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:09.723101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:09.726145 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:09.750135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 291/577] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:09.751662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:09.855505 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33edd2a-f61e-4b94-9f09-57fd3ed7f93b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.904132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.904922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.905283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.906152 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.918743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:09.920335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:09.933901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161936f-fee0-42c7-898d-7e9b1663375c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:09.937113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:09.938903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:09.941555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:09.941555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:09.941555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:09.944122 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 287/578] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:09.945019 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:09.961799 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:09.991352 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:09.992384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:09.992878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:09.993347 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:09.995140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:09.996050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:09.998672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.021340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 292/579] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:10.023045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:10.102066 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:10.106136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24bf64-63b3-47ac-b705-d0ea63b43487 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.163070 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:10.164023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:10.164824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:10.164824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:10.165769 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:10.166264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:10.168256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.182769 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 288/580] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:10.184302 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:10.204573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a35e94-0089-4724-a430-34504da301c1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.259911 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:10.260663 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:10.261005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:10.261354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:10.270434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:10.272036 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:10.284890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:10.286839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:10.289069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:10.290832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 293/581] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:10.291756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:10.306486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:10.348021 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62809319-00c6-4e5d-aeb1-1894f24e14ca None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.397595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:10.398242 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:10.398534 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:10.398800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:10.399453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:10.399750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:10.401147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.424706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 289/582] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:10.424706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:10.477953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec48e0-abb0-4591-9c39-9e5fecdb3712 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.496838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:10.539770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:10.540827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:10.541310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:10.541824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:10.557878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:10.560553 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:10.583866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:10.584906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 11:58:10.586340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:10.589955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 294/583] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:10.590919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:10.613325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fd747-f246-4191-b9cc-7976e24234a1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.674840 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:10.684239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:10.695082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:10.695082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:10.695082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:10.706066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:10.708884 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.710667 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d453a-89eb-4407-aa1a-aabcac5ae92a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:10.739630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:10.741486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:10.743736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:10.743736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:10.743736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:10.743736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 290/584] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:10.743736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:10.758123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:10.758739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:10.759016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:10.759295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:10.760018 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:10.760320 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:10.761923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.777214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 295/585] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:10.778119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:10.834244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:10.886786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb53e88-ec36-411c-a084-a2002f55ddf6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.941359 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:10.943111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:10.943611 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:10.944104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:10.963184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:10.966431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:10.970626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be38cc-c367-4fbb-a448-c176fa140b3c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:10.979202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:10.989030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 291/586] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:10.992432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.042373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.043409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.043894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.044372 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.045786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:11.046343 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:11.049052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:11.072497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:11.086771 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 296/587] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:11.086771 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.163525 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bef11-d0c7-4f97-acdb-aa46d9afc3bb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:11.198449 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:11.205479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.206128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.206405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.206673 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.207333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:11.207616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:11.209012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:11.216453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c455c8c-9f6e-4c3f-a7e4-a2292bca9f82 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:11.224176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 292/588] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:11.224875 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.265842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.266423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.266760 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.267039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/0c8225879a66415c8a1dc28584e4c626/roles/bc60629774714a078250dbc704217d2a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.283374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=0c8225879a66415c8a1dc28584e4c626, role_id=bc60629774714a078250dbc704217d2a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:11.284844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:11.358530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60a938-7bee-44fe-b0db-9044f9c8adce None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:11.361116 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:11.391622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 297/589] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:11 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/0c8225879a66415c8a1dc28584e4c626/roles/bc60629774714a078250dbc704217d2a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:58:11.392922 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.416785 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.417100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.417457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.417748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.418847 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:11.419323 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:11.421645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:11.449756 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 293/590] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Sep 1 11:58:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:11.450995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.598790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443dba7-72df-4847-9b6c-ba9b2e106d88 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:11.619587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a67f30-0d2e-43a7-80f3-da98f6f2671e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:11.656405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.657058 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.657330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.657598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.666586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:11.668077 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:11.681723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:11.683710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 294/591] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:11.685487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.695527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ac2a2-dc08-4fe7-bd8e-eebd3ffbdb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.696278 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac2a2-dc08-4fe7-bd8e-eebd3ffbdb49 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.696627 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac2a2-dc08-4fe7-bd8e-eebd3ffbdb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.696992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac2a2-dc08-4fe7-bd8e-eebd3ffbdb49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.701147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:11.712879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ac2a2-dc08-4fe7-bd8e-eebd3ffbdb49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:11.715669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ac2a2-dc08-4fe7-bd8e-eebd3ffbdb49 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:11.727208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-f63ac2a2-dc08-4fe7-bd8e-eebd3ffbdb49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:11.745839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:11.747944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:11.750015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:11.750015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:11.750015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:11.757453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 298/592] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:11.760505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a14d2-07ac-415a-be81-bba5c2ecc66b None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.760973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a14d2-07ac-415a-be81-bba5c2ecc66b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.761450 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a14d2-07ac-415a-be81-bba5c2ecc66b None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.764095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 299/593] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:11.768765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a14d2-07ac-415a-be81-bba5c2ecc66b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.835271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd49d-2033-4e22-b180-f39b47fc982c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:11.881832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.882394 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.883019 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:11.883019 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:11.894835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:11.897363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:11.914716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:11.916447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:11.918205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:11.918205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:11.918205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:11.918205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 295/594] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:11.918205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:11.949983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a14d2-07ac-415a-be81-bba5c2ecc66b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:11.998369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60472e-abef-4203-9725-8ef7a48d3f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:11.999472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60472e-abef-4203-9725-8ef7a48d3f82 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:11.999752 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60472e-abef-4203-9725-8ef7a48d3f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.000028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60472e-abef-4203-9725-8ef7a48d3f82 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.000768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60472e-abef-4203-9725-8ef7a48d3f82 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:12.001066 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60472e-abef-4203-9725-8ef7a48d3f82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:12.007005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60472e-abef-4203-9725-8ef7a48d3f82 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:12.016891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:12.030929 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 300/595] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Sep 1 11:58:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:12.032374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce8e04-6542-42d3-8b50-90621419a2ed None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.032758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce8e04-6542-42d3-8b50-90621419a2ed None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.033135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce8e04-6542-42d3-8b50-90621419a2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.053827 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1bb10-fc65-4b58-bf2d-37852694d150 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:12.056958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-43ce8e04-6542-42d3-8b50-90621419a2ed None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:12.058546 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-43ce8e04-6542-42d3-8b50-90621419a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:12.105487 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:12.112866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:12.113941 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.114343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.114619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.115798 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:12.116348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:12.118958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:12.133027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 301/596] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:12.134272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce8e04-6542-42d3-8b50-90621419a2ed None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.145982 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 296/597] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:12.147403 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.289316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf613a-7a12-4d38-b061-f568b8a4376e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:12.291446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce8e04-6542-42d3-8b50-90621419a2ed None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:12.335432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:12.336359 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.336785 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.337235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.353221 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:12.355184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:12.356669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:12.364614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:12.366895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.366895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.366895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.368322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:12.369089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:12.370740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:12.374687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:12.375700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:12.376642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:12.376642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:12.376642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:12.380762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 297/598] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:12.381777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.401749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 302/599] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:12.401749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.453078 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:12.552465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbb512-5d90-488d-bd95-312d16c2c9b9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:12.568205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca9bc6-227e-4ac3-95e2-2dafd76eff12 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:12.603701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:12.604436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.604945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.605430 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.606954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:12.607513 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:12.611001 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:12.616735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:12.617440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.617736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.617995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.629721 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:12.631718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 303/600] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:12.632481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:12.633976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:12.657030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:12.658880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:12.660461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:12.661810 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 298/601] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:12.663642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.729941 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:12.783797 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:12.790444 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d222-6dcf-4683-b8d1-65b289aea6b5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:12.798835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479b92-67b3-49f9-bc2c-a4e5871939ad None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:12.835661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:12.836293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.836573 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.836845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.852367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:12.857808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:12.858906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 299/602] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:12.859918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.916652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:12.917790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:12.918259 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:12.918575 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:12.927267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:12.929268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:12.943643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:12.946015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 304/603] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:12.947850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:12.994287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f175-108a-40f5-a525-00ecf9bc91af None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.002713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:13.031964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7190831-8622-465b-9c79-308041786e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.032425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7190831-8622-465b-9c79-308041786e86 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.032768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7190831-8622-465b-9c79-308041786e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.033052 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7190831-8622-465b-9c79-308041786e86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.042768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7190831-8622-465b-9c79-308041786e86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.044297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7190831-8622-465b-9c79-308041786e86 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b7190831-8622-465b-9c79-308041786e86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:13.057305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:13.058473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:13.059591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:13.063806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 300/604] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:13.064718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7190831-8622-465b-9c79-308041786e86 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:13.097979 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64203645-90d5-458f-9a77-eb519bfdbf63 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.149240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.150114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.150449 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.150775 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.151842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:13.152336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.154892 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.182973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 305/605] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:13.182973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:13.208599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7190831-8622-465b-9c79-308041786e86 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.213236 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:13.247623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.248345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.248706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.249639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.250411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:13.250780 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.252241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.261416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 301/606] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:13.262572 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:13.297364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:13.348021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318a560-ec2d-4167-ba69-eac1a907df33 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.409902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.410923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.411367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.411782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.412534 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:13.413034 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.415580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.422985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdb69a-8931-4b46-8d03-338b63f267c4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.437501 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 306/607] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:13.438933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:13.471841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.472599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.473007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.473379 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.474598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:13.475166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.477423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.495285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 302/608] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:13.496279 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:13.563556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:13.594724 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:13.603668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b1dc5-f40c-4ee8-a8f7-bbc2c9e6738f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.607066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb05b47-0936-4fcb-b43e-928a9dcd6dd1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.642150 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.642981 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.643349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.643731 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/bb79770a161740128e5792a70cbad00a/roles/bc60629774714a078250dbc704217d2a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.645311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad01d33-b0ba-4db1-bc85-42282f7bd853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.646093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad01d33-b0ba-4db1-bc85-42282f7bd853 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.646453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad01d33-b0ba-4db1-bc85-42282f7bd853 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.647016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad01d33-b0ba-4db1-bc85-42282f7bd853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.660344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad01d33-b0ba-4db1-bc85-42282f7bd853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.661850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad01d33-b0ba-4db1-bc85-42282f7bd853 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.662032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=bb79770a161740128e5792a70cbad00a, role_id=bc60629774714a078250dbc704217d2a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.663468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-fad01d33-b0ba-4db1-bc85-42282f7bd853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:13.678110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:13.684958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 303/609] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231a63b-d9ce-46a4-9a7e-e26970190b2e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231a63b-d9ce-46a4-9a7e-e26970190b2e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231a63b-d9ce-46a4-9a7e-e26970190b2e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 304/610] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:13.687570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231a63b-d9ce-46a4-9a7e-e26970190b2e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:13.750195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 307/611] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:13 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/bb79770a161740128e5792a70cbad00a/roles/bc60629774714a078250dbc704217d2a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:58:13.751143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:13.816207 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231a63b-d9ce-46a4-9a7e-e26970190b2e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.875051 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.876113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.876613 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.877165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/86e8ec32ae1f46e9a1a3ada4fc7201c3/roles/bc60629774714a078250dbc704217d2a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.890473 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94970373-fccd-4fa3-8f9b-885c3a137e83 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:13.897311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:13.903505 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=86e8ec32ae1f46e9a1a3ada4fc7201c3, role_id=bc60629774714a078250dbc704217d2a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.906663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:13.928205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:13.947619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af752ec-16f7-45bb-b78c-f8f04642d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:13.947619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af752ec-16f7-45bb-b78c-f8f04642d3ee None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:13.947619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af752ec-16f7-45bb-b78c-f8f04642d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:13.947619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af752ec-16f7-45bb-b78c-f8f04642d3ee None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/75e0ceaa2c6745dc89ade0c43a17572b/roles/bc60629774714a078250dbc704217d2a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:13.965583 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af752ec-16f7-45bb-b78c-f8f04642d3ee None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=75e0ceaa2c6745dc89ade0c43a17572b, role_id=bc60629774714a078250dbc704217d2a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:13.967811 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af752ec-16f7-45bb-b78c-f8f04642d3ee None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:14.041471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 305/612] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:13 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/86e8ec32ae1f46e9a1a3ada4fc7201c3/roles/bc60629774714a078250dbc704217d2a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:58:14.042526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:14.090601 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 308/613] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:13 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/75e0ceaa2c6745dc89ade0c43a17572b/roles/bc60629774714a078250dbc704217d2a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:58:14.092788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9f04b-0c58-4824-b3eb-acd8318a9cc8 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:14.093492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9f04b-0c58-4824-b3eb-acd8318a9cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:14.093982 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9f04b-0c58-4824-b3eb-acd8318a9cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:14.127206 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-45d9f04b-0c58-4824-b3eb-acd8318a9cc8 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:14.130921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-45d9f04b-0c58-4824-b3eb-acd8318a9cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:14.259348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a14447-9d06-4799-96de-395cd96d409b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:14.275931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 309/614] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:14.328131 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:14.329897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:14.330361 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:14.331002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:14.331899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:14.332402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:14.334226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:14.355956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 306/615] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:14.361855 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9f04b-0c58-4824-b3eb-acd8318a9cc8 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:14.567642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9f04b-0c58-4824-b3eb-acd8318a9cc8 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:14.582871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:14.635013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:14.636130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:14.636634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:14.637142 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:14.652383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:14.655753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:14.680074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:14.681125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:14.682133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:14.682702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 310/616] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:14.688726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:14.717378 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:14.784715 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:14.831758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e605a1-9df9-47d1-9327-be87523a5fea None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:14.870024 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:14.871153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:14.871632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:14.872212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:14.885529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:14.887946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f019dd-6b73-4621-8cea-fc50bb7c3a9f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:14.892700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:14.893877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:14.894784 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:14.897561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 311/617] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:14.903673 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:14.934770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:14.999959 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:15.000596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:15.000945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:15.001148 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:15.001905 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:15.002214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:15.003692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:15.021353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f544f0-b6d1-404d-b6a5-63d45aa493fa None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:15.043908 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 307/618] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Sep 1 11:58:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:15.051947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:15.066548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:15.067463 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:15.068001 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:15.068449 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:15.069536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:15.070012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:15.071743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:15.082554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 312/619] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:15.088505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:15.190228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:15.206668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae8b4-dc94-46d1-a457-75aaeb108797 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:15.258453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:15.259858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:15.260506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:15.261174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:15.274127 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:15.276003 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:15.293650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:15.294671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:15.295570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:15.295570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:15.295570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:15.298533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 313/620] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:15.304774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:15.331309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9d74-b19f-4cb2-896e-56408281a683 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:15.352828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:15.406544 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:15.407191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:15.407470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:15.407800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:15.426680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43c0f-654d-461a-a725-51ef8bfe7548 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:15.428418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:15.429945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:15.460818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:15.461896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:15.462832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:15.462832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:15.462832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:15.469221 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 308/621] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:15.470494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:15.481520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:15.482668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:15.483195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:15.483740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:15.484608 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:15.485119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:15.486834 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:15.501580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 314/622] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:15.501580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:15.507838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:15.662748 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0d15e-250a-44da-88eb-164a2133bc52 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:15.722000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e255081-34c2-4e56-b1b5-71265d18d838 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:15.743058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:15.743723 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:15.744008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:15.744292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:15.757244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:15.758718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:15.775747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:15.776875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:15.777876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:15.778472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:15.778565 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 315/623] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:58:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:15.778644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:15.778644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:15.778644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:15.779423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:15.779423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:15.781176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:15.787517 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:15.800861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 309/624] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:15.804484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:15.841867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:15.900764 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:16.004544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1719e5-b3e6-4ea4-a5de-0e3200c84f2e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:16.013205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a587fd-c0f5-422a-a78a-85adf3959a62 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:16.067786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:16.068432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.069332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.069719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.078018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:16.079593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:16.109468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:16.110495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:16.111391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:16.112318 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 310/625] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:16.119251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:16.139381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:16.140494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.140989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.141471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.142488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:16.143235 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:16.144943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:16.179205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 316/626] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:58:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:16.188158 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:16.325161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:16.350138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce6559-97c1-4390-a424-27e6695670df None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:16.400747 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:16.401778 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.402290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.402770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.403701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:16.404255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:16.407685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:16.427025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134af15-a725-461c-b9fc-461964bda03b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:16.446036 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 311/627] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:16.461485 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:16.486225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:16.539024 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0c093-cfd0-4fd6-bc91-a4e385329f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:16.539658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0c093-cfd0-4fd6-bc91-a4e385329f5c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.539940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0c093-cfd0-4fd6-bc91-a4e385329f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.540217 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0c093-cfd0-4fd6-bc91-a4e385329f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.594402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692ef06-50cd-4826-999b-3d8ea144e004 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:16.653138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0c093-cfd0-4fd6-bc91-a4e385329f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:16.654812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0c093-cfd0-4fd6-bc91-a4e385329f5c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:16.660674 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f7edc-8b0e-4b7c-b8aa-74a0f6e545e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:16.661853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f7edc-8b0e-4b7c-b8aa-74a0f6e545e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.662324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f7edc-8b0e-4b7c-b8aa-74a0f6e545e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.662826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f7edc-8b0e-4b7c-b8aa-74a0f6e545e6 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/60c4eccc7fb0434e9251fa1b9bd7aec2/roles/bc60629774714a078250dbc704217d2a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.683923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:16.691318 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f7edc-8b0e-4b7c-b8aa-74a0f6e545e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=60c4eccc7fb0434e9251fa1b9bd7aec2, role_id=bc60629774714a078250dbc704217d2a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:16.694419 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f7edc-8b0e-4b7c-b8aa-74a0f6e545e6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:16.760951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 317/628] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:16.762290 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041867-4eb8-4676-9e99-d610b98a4305 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.762577 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041867-4eb8-4676-9e99-d610b98a4305 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.762849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041867-4eb8-4676-9e99-d610b98a4305 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.764261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 318/629] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:16.765842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba60c-4626-42cd-b561-0a98edeb8437 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.766410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba60c-4626-42cd-b561-0a98edeb8437 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.766410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba60c-4626-42cd-b561-0a98edeb8437 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.767678 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 319/630] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:16.768749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a0f3-8cbf-4434-918b-5818066a2875 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.769236 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a0f3-8cbf-4434-918b-5818066a2875 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.769353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424a0f3-8cbf-4434-918b-5818066a2875 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.770548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 320/631] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:16.771366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424a0f3-8cbf-4434-918b-5818066a2875 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:16.788872 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 312/632] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:16 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/60c4eccc7fb0434e9251fa1b9bd7aec2/roles/bc60629774714a078250dbc704217d2a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:58:16.791160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89488df-3e89-4f1d-ad95-daebd3b7ccea None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.791677 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89488df-3e89-4f1d-ad95-daebd3b7ccea None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.792182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89488df-3e89-4f1d-ad95-daebd3b7ccea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.792611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:16.825362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e89488df-3e89-4f1d-ad95-daebd3b7ccea None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:16.827688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e89488df-3e89-4f1d-ad95-daebd3b7ccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:16.895504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424a0f3-8cbf-4434-918b-5818066a2875 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:16.934732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284840a-a241-405d-8500-3f8c9a7feaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:16.935385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284840a-a241-405d-8500-3f8c9a7feaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.935667 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284840a-a241-405d-8500-3f8c9a7feaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.935938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284840a-a241-405d-8500-3f8c9a7feaf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.951102 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284840a-a241-405d-8500-3f8c9a7feaf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:16.953254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284840a-a241-405d-8500-3f8c9a7feaf9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:16.963574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 313/633] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:16.964903 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614028ce-2ceb-4553-a5c3-5a99318c172a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.965177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614028ce-2ceb-4553-a5c3-5a99318c172a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.965467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614028ce-2ceb-4553-a5c3-5a99318c172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a284840a-a241-405d-8500-3f8c9a7feaf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:16.972952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:16.974008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:16.974957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:16.974957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:16.974957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:16.974957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 321/634] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:16.975913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7f88-cd46-47b3-a536-66d339d4e6cd None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:16.976201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7f88-cd46-47b3-a536-66d339d4e6cd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:16.976479 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7f88-cd46-47b3-a536-66d339d4e6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:16.996403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-838d7f88-cd46-47b3-a536-66d339d4e6cd None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:16.997676 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-838d7f88-cd46-47b3-a536-66d339d4e6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:16.997996 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-614028ce-2ceb-4553-a5c3-5a99318c172a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:17.000811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-614028ce-2ceb-4553-a5c3-5a99318c172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:17.075957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:17.078976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 322/635] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:17.080429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7f88-cd46-47b3-a536-66d339d4e6cd None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:17.167098 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 314/636] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:17.242931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7f88-cd46-47b3-a536-66d339d4e6cd None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:17.297642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:17.298285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:17.298567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:17.298842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:17.299507 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:17.299815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:17.301260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:17.316343 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 323/637] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:17.321967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614028ce-2ceb-4553-a5c3-5a99318c172a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:17.366075 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:17.399666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:17.473714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:17.492724 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614028ce-2ceb-4553-a5c3-5a99318c172a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:17.535226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:17.536065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:17.536574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:17.536954 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:17.546283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:17.548004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:17.560829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb167670-9b22-4184-8551-376d3ea4c96b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:17.562234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:17.564121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:17.566093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:17.570414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 315/638] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:17.571465 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:17.640407 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:17.642054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:17.642873 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:17.643617 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:17.666050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:17.677814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:17.679969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:17.682005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:17.686797 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 324/639] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:17.688779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:17.689501 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c268a5-709d-44dc-adf3-9591a481f34e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:17.703523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:17.754185 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:17.755724 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:17.756188 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:17.756664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:17.777423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:17.788953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:17.802949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 316/640] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:17.804617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:17.857704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897b56c-b37d-4a6c-994f-d9eb1058cc9a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:17.884845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:17.901360 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:17.902389 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:17.902773 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:17.903175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:17.904036 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:17.904478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:17.906907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:17.926330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 325/641] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:17.927786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:17.955435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce871b-82c6-4cb0-bf15-77f546fd85be None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:18.007690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:18.008774 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:18.009278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:18.009807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:18.011099 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:18.011649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:18.014432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:18.033717 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 317/642] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:18.033717 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:18.074451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58453e2a-5705-4d9e-9bc7-cbbd4e93ef25 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:18.099405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:18.125034 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:18.126060 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:18.126523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:18.126966 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:18.146101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:18.154644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:18.157426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 326/643] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:18.160443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:18.223970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbeb910-c63f-4cc1-baae-98fc4dac1c9b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:18.240569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:18.274170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:18.274998 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:18.275401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:18.275866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:18.276761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:18.277136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:18.278803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:18.291070 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 318/644] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:18.292838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:18.368198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e4789-f727-4d47-8fa6-49df7f9b8bf5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:18.421080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:18.430677 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393148a1-24c3-4aa4-b956-79d595dcc7c4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:18.437874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:18.438918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:18.444871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:18.444871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:18.444871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=0c8225879a66415c8a1dc28584e4c626) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:18.444871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:18.444871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:18.466232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 327/645] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:18 2022] GET /identity/v3/role_assignments?user.id=0c8225879a66415c8a1dc28584e4c626&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:18.466232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:18.515224 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:18.516431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:18.516735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:18.517015 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:18.539686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:18.542650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:18.577626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:18.578865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:18.580630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:18.580630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:18.580630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:18.585041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 319/646] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:18.586073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:18.692802 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8b71c-aafb-40aa-ab1e-8bf717cd7041 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:18.698139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:18.757522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:18.758640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:18.759101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:18.759627 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:18.760825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:18.761374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:18.764020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:18.776677 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b1eab-14c1-4f51-ac20-d40710b870fe None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:18.782475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 328/647] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:18.783962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:18.837066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:18.838375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:18.838961 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:18.839804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:18.866271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:18.869780 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:18.886164 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:18.910467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:18.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:18.912399 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:18.912399 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:18.912399 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:18.913005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 320/648] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:18.914428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.010730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46c9a7-8d12-4412-8a25-2990a1598c61 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:19.083560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:19.084629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.085122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.085650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.086860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:19.087582 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:19.090198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:19.101950 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14cb42-9639-47da-94f4-289c1f272a8d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:19.118469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 329/649] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:19.125661 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.154455 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac497b88-348d-46a3-ac5e-4b16007acb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:19.155832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac497b88-348d-46a3-ac5e-4b16007acb37 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.156109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac497b88-348d-46a3-ac5e-4b16007acb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.156402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac497b88-348d-46a3-ac5e-4b16007acb37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.158236 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:19.169317 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac497b88-348d-46a3-ac5e-4b16007acb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:19.174051 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac497b88-348d-46a3-ac5e-4b16007acb37 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ac497b88-348d-46a3-ac5e-4b16007acb37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:19.197434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:19.199178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:19.201589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:19.201589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:19.201589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:19.201589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 321/650] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:19.201589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6470db-8b83-470f-b626-2f3980928487 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.201589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6470db-8b83-470f-b626-2f3980928487 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.203122 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6470db-8b83-470f-b626-2f3980928487 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.209989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 322/651] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:19.211049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6470db-8b83-470f-b626-2f3980928487 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.280367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:19.315353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28818d1d-46bd-48b4-89c6-97c7b4cd4e7c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:19.358290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6470db-8b83-470f-b626-2f3980928487 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:19.377168 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:19.378258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.378747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.379242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.380418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:19.380984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:19.383618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:19.405190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937b124-167c-45c6-80e4-261f236d778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:19.406304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b124-167c-45c6-80e4-261f236d778f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.406831 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b124-167c-45c6-80e4-261f236d778f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.407328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b124-167c-45c6-80e4-261f236d778f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.414665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 330/652] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:19.414665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.424059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937b124-167c-45c6-80e4-261f236d778f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:19.426352 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937b124-167c-45c6-80e4-261f236d778f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0937b124-167c-45c6-80e4-261f236d778f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:19.440932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:19.442946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 11:58:19.445265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:19.448478 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 323/653] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:19.450683 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec8d00-c71e-48d6-a589-33f7bf193af5 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.451427 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec8d00-c71e-48d6-a589-33f7bf193af5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.452851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec8d00-c71e-48d6-a589-33f7bf193af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.475239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a7ec8d00-c71e-48d6-a589-33f7bf193af5 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:19.476638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a7ec8d00-c71e-48d6-a589-33f7bf193af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:19.568028 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 324/654] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:19.570208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec8d00-c71e-48d6-a589-33f7bf193af5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.615864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:19.648883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:19.656782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4863a4-ea48-4458-a1d4-fa992779a7b7 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:19.724316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:19.725363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.725363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.725829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.726350 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:19.726773 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:19.728623 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:19.748396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec8d00-c71e-48d6-a589-33f7bf193af5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:19.753969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 331/655] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:19.756071 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.802518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:19.803356 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.803690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.804081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.815073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:19.817027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:19.833007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:19.834922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 325/656] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:19.837743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.921496 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17602ba2-d15a-4f0b-b7b0-35dc13cf6aa5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:19.960547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:19.961168 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:19.961451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:19.961758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:19.962502 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:19.962809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:19.964333 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:19.984004 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 332/657] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Sep 1 11:58:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:19.984912 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:19.989113 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.048610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.054029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd475f5a-936f-44fe-8a8a-143719d753b8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:20.116740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:20.118199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.118886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.119657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.130160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:20.131888 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:20.145073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:20.146243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 11:58:20.147344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:20.149256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 326/658] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:20.150276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:20.175679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba77c2-8a13-45ec-ac43-e988e95f9c78 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:20.232598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:20.234146 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.234543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.234983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.236274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:20.236777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:20.239122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:20.273891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 333/659] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Sep 1 11:58:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:20.273891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:20.343627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918cbdf-3679-4344-9ac6-def01ee69415 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:20.347863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.396985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:20.397763 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.398121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.398586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.406402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4c3e7-6b7d-4e94-9290-c22d8e082198 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:20.434856 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.459134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:20.459865 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.460253 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.460529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.476718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:20.484651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:20.486172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 334/660] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:20.487141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:20.497414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:20.499458 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:20.622218 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c421a-9005-40cc-a0bc-f3047cf2178d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:20.631067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 327/661] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:20.631067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:20.648509 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.662567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:20.663432 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.663725 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.664101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.665199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:20.665734 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:20.667998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:20.693650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 335/662] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Sep 1 11:58:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:20.694745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:20.700722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.757979 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4331a0-1b8d-43dc-ae46-432fd07f9150 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:20.803842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:20.804471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.804759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.805069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.824890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd3120-30c0-4789-9f8f-7cc2a7ecffb5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:20.865943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14cd4-089d-4b09-bcd3-0caacf512c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:20.866571 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14cd4-089d-4b09-bcd3-0caacf512c19 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.866924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14cd4-089d-4b09-bcd3-0caacf512c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.867436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14cd4-089d-4b09-bcd3-0caacf512c19 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.869056 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14cd4-089d-4b09-bcd3-0caacf512c19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:20.869377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14cd4-089d-4b09-bcd3-0caacf512c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:20.871336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14cd4-089d-4b09-bcd3-0caacf512c19 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:20.881160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:20.882660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 336/663] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:20.882788 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:20.884003 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc931a-d268-4b36-b29a-f3f88f1f383e None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:20.884358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc931a-d268-4b36-b29a-f3f88f1f383e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:20.884641 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc931a-d268-4b36-b29a-f3f88f1f383e None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:20.886523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 337/664] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:20.887288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc931a-d268-4b36-b29a-f3f88f1f383e None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:20.942519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.955504 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:20.997673 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 328/665] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:20.998637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.005148 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc931a-d268-4b36-b29a-f3f88f1f383e None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:21.049906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:21.050566 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.050849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.051125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.063864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:21.065358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:21.083272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:21.085197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:21.087100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:21.087100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.087100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:21.087100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 338/666] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:21.087100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.163494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348bcd-1ec8-4d40-85df-27071a727443 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:21.208879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:21.209266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:21.209492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.210108 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.210434 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.212542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ffe4b-8f81-4355-a697-247378cb7c4c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:21.246123 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:21.257921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafab2ca-3afc-4cd3-9e98-e09998b25f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:21.258817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafab2ca-3afc-4cd3-9e98-e09998b25f10 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.259238 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafab2ca-3afc-4cd3-9e98-e09998b25f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.259559 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafab2ca-3afc-4cd3-9e98-e09998b25f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.274599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafab2ca-3afc-4cd3-9e98-e09998b25f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:21.276193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafab2ca-3afc-4cd3-9e98-e09998b25f10 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-eafab2ca-3afc-4cd3-9e98-e09998b25f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:21.293816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:21.296200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:21.297986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:21.297986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.297986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:21.297986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 339/667] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:21.297986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d7f03-aac2-40fb-bb16-8ee25b4438c0 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.297986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d7f03-aac2-40fb-bb16-8ee25b4438c0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.297986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d7f03-aac2-40fb-bb16-8ee25b4438c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.311338 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:21.313384 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:21.317139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-490d7f03-aac2-40fb-bb16-8ee25b4438c0 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:21.318398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-490d7f03-aac2-40fb-bb16-8ee25b4438c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:21.409293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 340/668] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:21.410433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d7f03-aac2-40fb-bb16-8ee25b4438c0 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.445758 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 329/669] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:21.445758 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.524148 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:21.554093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:21.554873 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d7f03-aac2-40fb-bb16-8ee25b4438c0 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:21.612973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:21.613965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.614422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.614871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.621557 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426326de-79fe-4cc6-9444-87ef2e179c43 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:21.630826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:21.632476 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:21.650942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:21.652796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:21.655939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:21.655939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.655939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:21.655939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 341/670] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:21.655939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.673973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:21.674575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.674856 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.675512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.675777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:21.676081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:21.677501 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:21.690520 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 330/671] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:21.691152 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.802710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605bad7-168c-4ba6-bda8-1959442f13b0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:21.810256 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc2768-a27f-4e8d-bf30-6b0e8b69247b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:21.824409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:21.838576 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:21.839816 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:21.840451 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.841486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.841486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.846349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:21.846965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:21.847248 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:21.847522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:21.848252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:21.848996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:21.849955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:21.852446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:21.853930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:21.863806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 342/672] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:21.864009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:21.870605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:21.871707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:21.872990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:21.872990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:21.872990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:21.872990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 331/673] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:21.872990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:21.981628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6a0f5-f269-4216-bbb7-0f1b018a06c6 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.005526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1bf4d-c90f-451a-9a44-2a38c888a4a3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.020471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.021092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.021373 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.021670 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.022334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:22.022640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.024035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.036968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 343/674] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:22.038100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.042409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.043055 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.043339 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.043616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.057586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:22.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:22.065804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 332/675] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:22.067608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.100086 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.104010 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.159404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de62bca-1c0e-4278-a901-e78774ab1820 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.179124 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f7ab-11c1-49e5-89ee-16122d379c3e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.201310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.201983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.202801 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.203081 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.210767 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.212224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.218166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.218808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.219183 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.219441 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:22.224619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:22.226335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 344/676] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:22.228997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.239839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.241343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:22.254149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:22.255336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:22.256523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:22.260161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 333/677] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:22.261078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.364165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18ae2c-6bb7-4b79-aa13-91e7e7362857 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.365421 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.374323 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.379065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53094711-2d65-421e-9773-3f35a1731cd3 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.404552 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.405746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.406033 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.406312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.406970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:22.407276 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.408665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.422596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 345/678] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:22.423487 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.425389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.426201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.426704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.427208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.438540 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.441319 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:22.458209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:22.460942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 334/679] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:22.462939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.541203 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa8910-147e-4f44-beec-50de1ca41abc None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.585941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.586561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.586846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.587125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.587797 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:22.588106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.589482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.599409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 346/680] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:22.600266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.607814 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f16ca-03c5-4d4f-80df-6e9576ced3d5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.635356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.646311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.646925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.648984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.648984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.648984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:22.648984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.649901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.663509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 335/681] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:22.664379 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.681942 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.728741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4449-d5f7-49bb-a560-55f6aa9da41d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.774857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.775891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.776488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.776958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.778278 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:22.778735 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.780429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.783952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43046a46-b45c-4ae0-838b-87544a49c6c4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.800367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 347/682] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:22.801238 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.831547 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.832205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.832488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.832820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.840430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.841932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:22.854165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:22.855627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 336/683] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:22.857198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.913892 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.927163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774e3c-8efb-4803-8112-039b7e20f9d3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:22.950194 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:22.966472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:22.967097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:22.967383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:22.967665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:22.968377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:22.968688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:22.970188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:22.984339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 348/684] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:22.985469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:22.998038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf929-89e4-4cc5-9f42-e8af5005d611 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:23.043045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:23.043734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:23.044007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:23.044285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:23.057945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:23.059432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:23.074685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:23.075777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:23.076751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:23.076751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:23.076751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:23.076751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 337/685] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:23.076751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:23.107346 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49d01a-7c8e-4907-8d68-01791f819e7c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:23.146123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:23.146739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:23.147022 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:23.147310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:23.148026 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=bb79770a161740128e5792a70cbad00a) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:23.148332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:23.149796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:23.156348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 349/686] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:22 2022] GET /identity/v3/role_assignments?user.id=bb79770a161740128e5792a70cbad00a&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:23.157159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:23.187214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:23.189068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9af97-1d75-43cf-a2e8-a9975e41c0cd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:23.217179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:23.232913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:23.234529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:23.234910 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:23.235279 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:23.297483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d58c-92ea-4a65-924f-91ff9f1d72bf None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:23.345887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:23.347361 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:23.355134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:23.356042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:23.356588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:23.357043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:23.358371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:23.358920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:23.361780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:23.376963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 350/687] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:23.377937 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:23.467461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:23.489690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 338/688] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:23.490795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:23.512263 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa8be4-f482-46db-b6fe-fecf0a36db50 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:23.522676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:23.557742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:23.558711 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:23.559143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:23.559586 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:23.560701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=86e8ec32ae1f46e9a1a3ada4fc7201c3) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:23.561257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:23.564332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:23.574482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 351/689] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:23 2022] GET /identity/v3/role_assignments?user.id=86e8ec32ae1f46e9a1a3ada4fc7201c3&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:23.575535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:23.658112 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9717fb-110b-4e55-a36a-3b73e923bfe0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:23.710027 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:23.711087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:23.712684 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:23.713250 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:23.714280 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:23.714821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:23.716559 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:23.732055 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 339/690] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:23.733925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:23.757414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec050c-3d75-4091-85aa-31d1fd3aea49 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:23.815581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:23.816610 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:23.817692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:23.818201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:23.819888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=75e0ceaa2c6745dc89ade0c43a17572b) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:23.820448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:23.822446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:23.831371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 352/691] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:23 2022] GET /identity/v3/role_assignments?user.id=75e0ceaa2c6745dc89ade0c43a17572b&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:23.832549 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:23.838003 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:23.922677 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:23.970220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171d802-f3bf-4594-9fef-b56385b5a9fc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:24.022003 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:24.023085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:24.023587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:24.024127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:24.038247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:24.040375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:24.061396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:24.062518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:24.063453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:24.063453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:24.063453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:24.065726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 340/692] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:24.074949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:24.095221 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73dfc6-8f1b-487b-89c9-238f42f13a32 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:24.138040 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:24.138684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:24.138959 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:24.139228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:24.148696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:24.150320 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:24.164398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:24.165451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:24.166690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 353/693] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:58:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:24.172372 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:24.236960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec3090-9d8d-4fcf-b317-b61477d448c6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:24.239198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:24.280563 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:24.281573 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:24.282134 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:24.284166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:24.285818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:24.286532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:24.289535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:24.294786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:24.316839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 341/694] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:24.323085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:24.342271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbb37f-abdb-4da3-9cae-35e51d4397ec None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:24.434515 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:24.435519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:24.436368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:24.436850 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:24.438131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:24.438662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:24.445060 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:24.488190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 354/695] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:58:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:24.494180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:24.576373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17445d4d-0b95-4a7e-8073-18f1ad7e7e59 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:24.625320 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182df25-8166-40b9-939f-7308b919848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:24.626330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182df25-8166-40b9-939f-7308b919848a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:24.626835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182df25-8166-40b9-939f-7308b919848a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:24.627289 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182df25-8166-40b9-939f-7308b919848a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:24.642096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182df25-8166-40b9-939f-7308b919848a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:24.644958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182df25-8166-40b9-939f-7308b919848a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4182df25-8166-40b9-939f-7308b919848a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:24.672084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:24.673097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:24.674038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:24.677767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 342/696] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:24.679314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182df25-8166-40b9-939f-7308b919848a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:24.701148 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7f542-98df-4d11-9cbf-a8cbd0d903cf None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:24.745983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:24.747011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:24.748285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:24.749335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:24.766032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:24.848663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:24.888931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:24.891915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:24.961541 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182df25-8166-40b9-939f-7308b919848a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:25.046931 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:25.047527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:25.047815 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:25.048090 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:25.048797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:25.049104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:25.050602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:25.080342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 343/697] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:25.088332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:25.116466 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 355/698] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:25.130725 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:25.151155 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:25.310311 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3232c-ff23-4f93-9e3a-964e5afaa658 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:25.377159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:25.377860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:25.378141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:25.378414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:25.387920 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69374d8-fc58-4978-b249-bb23352b081e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:25.392124 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:25.393915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:25.412120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:25.413568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:25.414538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:25.414538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:25.414538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:25.414538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 356/699] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:25.425043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:25.439894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:25.507812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:25.514996 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:25.515575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:25.515860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:25.516136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:25.516898 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=60c4eccc7fb0434e9251fa1b9bd7aec2) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:25.517234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:25.518737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:25.525986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 344/700] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:25 2022] GET /identity/v3/role_assignments?user.id=60c4eccc7fb0434e9251fa1b9bd7aec2&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 608 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:25.546454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979927bb-efa7-4229-b7b4-8cacb8d80f85 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:25.588007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515dc7b-e549-44d0-9db0-466e82664cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:25.588545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dc7b-e549-44d0-9db0-466e82664cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:25.588819 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dc7b-e549-44d0-9db0-466e82664cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:25.589092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dc7b-e549-44d0-9db0-466e82664cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:25.589780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515dc7b-e549-44d0-9db0-466e82664cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:25.590084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515dc7b-e549-44d0-9db0-466e82664cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:25.591483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515dc7b-e549-44d0-9db0-466e82664cd8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:25.607008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 357/701] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:25.615945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:25.668325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3fa05-a484-4304-875a-9daae7b53b78 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:25.668594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3fa05-a484-4304-875a-9daae7b53b78 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:25.668947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3fa05-a484-4304-875a-9daae7b53b78 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:25.671205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 358/702] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:25.683480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b855c-c89b-40fc-b742-cb29bc1dd3fb None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:25.683745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b855c-c89b-40fc-b742-cb29bc1dd3fb None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:25.684055 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b855c-c89b-40fc-b742-cb29bc1dd3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:25.723877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-894b855c-c89b-40fc-b742-cb29bc1dd3fb None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:25.725136 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-894b855c-c89b-40fc-b742-cb29bc1dd3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:25.794243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d34f1f-2d8f-426d-a2c5-8a0843207c5e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:25.842364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:25.842956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:25.843239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:25.843519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:25.852908 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:25.854458 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:25.873133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:25.874992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:25.876787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 345/703] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:25.888179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:25.890350 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 359/704] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:58:25.947834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:26.040484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66db573-71b7-4d7f-9edc-d162e51e6eea None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:26.113336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc966e-6922-40d5-95a6-47a1da2405b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:26.114354 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc966e-6922-40d5-95a6-47a1da2405b8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:26.114854 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc966e-6922-40d5-95a6-47a1da2405b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:26.115356 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc966e-6922-40d5-95a6-47a1da2405b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:26.116678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc966e-6922-40d5-95a6-47a1da2405b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:26.117240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc966e-6922-40d5-95a6-47a1da2405b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:26.120140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc966e-6922-40d5-95a6-47a1da2405b8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:26.148633 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 346/705] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:26.158624 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b855c-c89b-40fc-b742-cb29bc1dd3fb None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:26.213088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d900436-136e-4a5d-98b1-f3f4fa6c3de8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:26.213348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d900436-136e-4a5d-98b1-f3f4fa6c3de8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:26.213728 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d900436-136e-4a5d-98b1-f3f4fa6c3de8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:26.217000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 347/706] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:58:26.226483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1544ac-66f1-4ee3-a317-b70d80df002f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:26.226992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1544ac-66f1-4ee3-a317-b70d80df002f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:26.227493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1544ac-66f1-4ee3-a317-b70d80df002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:26.284252 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2f1544ac-66f1-4ee3-a317-b70d80df002f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:26.286324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2f1544ac-66f1-4ee3-a317-b70d80df002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:26.355806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:26.451004 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b855c-c89b-40fc-b742-cb29bc1dd3fb None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:26.485331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 348/707] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:26.488973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b1d83-3aa4-4e2d-be83-230edf7237fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:26.489596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b1d83-3aa4-4e2d-be83-230edf7237fd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:26.489904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b1d83-3aa4-4e2d-be83-230edf7237fd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:26.490176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b1d83-3aa4-4e2d-be83-230edf7237fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:26.490924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b1d83-3aa4-4e2d-be83-230edf7237fd None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=0c8225879a66415c8a1dc28584e4c626) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:26.491233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b1d83-3aa4-4e2d-be83-230edf7237fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:26.491931 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee7f1d-ece1-4c03-8645-64a74f891ea2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:26.492457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee7f1d-ece1-4c03-8645-64a74f891ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:26.492724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b1d83-3aa4-4e2d-be83-230edf7237fd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:26.492974 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee7f1d-ece1-4c03-8645-64a74f891ea2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:26.495610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 349/708] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:26.497091 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee7f1d-ece1-4c03-8645-64a74f891ea2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:26.504846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 360/709] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:26 2022] GET /identity/v3/role_assignments?user.id=0c8225879a66415c8a1dc28584e4c626&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:26.506160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414725d2-3870-439b-9eae-7f5b8d2ba554 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:26.506443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414725d2-3870-439b-9eae-7f5b8d2ba554 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:26.506728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414725d2-3870-439b-9eae-7f5b8d2ba554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:26.526268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-414725d2-3870-439b-9eae-7f5b8d2ba554 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:26.527568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-414725d2-3870-439b-9eae-7f5b8d2ba554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:26.559149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:26.607157 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 361/710] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:26.722624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee7f1d-ece1-4c03-8645-64a74f891ea2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:26.772047 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:26.773136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:26.773651 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:26.774122 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:26.791947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:26.793924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:26.813197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:26.814263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:26.815167 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:26.815167 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:26.815167 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:26.817509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 350/711] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:26.826832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414725d2-3870-439b-9eae-7f5b8d2ba554 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:26.897101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:26.963470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414725d2-3870-439b-9eae-7f5b8d2ba554 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:26.997198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:27.012466 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:27.013105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:27.013386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:27.013715 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:27.014940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:27.015260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:27.021639 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:27.032696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 362/712] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:27.041439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:27.159609 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:27.203377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f5689-5548-4b4b-aa51-02ec94331c4d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:27.204114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d018e4-2dbc-47d1-b62d-98482fdbc604 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:27.223135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:27.250081 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:27.250953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:27.251379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:27.251856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:27.252682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:27.254317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:27.283305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:27.284255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:27.284756 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:27.285243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] PATH_INFO: `/v3/services/network` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:27.286357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:27.288722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:27.301513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:27.302923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:27.303879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:27.306099 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 351/713] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:26 2022] GET /identity/v3/services/network => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:27.307127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:27.379944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 363/714] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:58:27 2022] POST /identity/v3/users => generated 254 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:27.381313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:27.567836 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011c408-051d-4cd3-b16a-aa5ea3020a86 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:27.617508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:27.618449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:27.618881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:27.619448 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:27.620041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:27.621636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:27.634430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:27.635714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:27.636657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:27.639197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 352/715] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Sep 1 11:58:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:27.644656 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:27.662669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:27.682704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580f17e-195c-4d07-8134-f71d3056f93a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:27.776365 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:27.777446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:27.778050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:27.778630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:27.780557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] RBAC: Adding query filter params (name=network) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:27.781109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:27.784125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:27.787244 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7285f-6e38-483c-b01f-17690b37a602 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:27.795441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 364/716] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:58:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:27.802532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:27.836881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:27.837912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:27.838387 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:27.838848 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:27.839706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:27.840231 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:27.842080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:27.848752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 353/717] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Sep 1 11:58:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:27.854402 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:27.894049 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:27.966497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:28.033331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c88a8-76d2-410d-b021-1e4212285fce None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:28.048493 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d9fc-605f-4752-b959-00b6a804591c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:28.131364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a6c28-8645-4e0a-abf2-2df732b05ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:28.133060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a6c28-8645-4e0a-abf2-2df732b05ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:28.133871 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a6c28-8645-4e0a-abf2-2df732b05ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:28.134668 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a6c28-8645-4e0a-abf2-2df732b05ed6 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:28.136137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a6c28-8645-4e0a-abf2-2df732b05ed6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:28.136964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a6c28-8645-4e0a-abf2-2df732b05ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:28.141066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a6c28-8645-4e0a-abf2-2df732b05ed6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:28.146421 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b6b2e-6cfe-49ea-b268-cfb162b9f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:28.147068 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b6b2e-6cfe-49ea-b268-cfb162b9f520 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:28.147351 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b6b2e-6cfe-49ea-b268-cfb162b9f520 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:28.147630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b6b2e-6cfe-49ea-b268-cfb162b9f520 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:28.148680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b6b2e-6cfe-49ea-b268-cfb162b9f520 None admin] RBAC: Adding query filter params (type=network) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:28.148680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b6b2e-6cfe-49ea-b268-cfb162b9f520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:28.150015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b6b2e-6cfe-49ea-b268-cfb162b9f520 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:28.154997 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 354/718] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Sep 1 11:58:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:28.171166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 365/719] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:58:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:28.190570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57c9f-f9ce-48e1-a26c-be3eac5bdce7 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:28.191539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57c9f-f9ce-48e1-a26c-be3eac5bdce7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:28.192373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57c9f-f9ce-48e1-a26c-be3eac5bdce7 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:28.195675 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 355/720] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:28.206311 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e051-a2ee-49f2-99e0-b2f792cfcfda None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:28.207891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e051-a2ee-49f2-99e0-b2f792cfcfda None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:28.208654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e051-a2ee-49f2-99e0-b2f792cfcfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:28.265828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-af85e051-a2ee-49f2-99e0-b2f792cfcfda None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:28.268355 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-af85e051-a2ee-49f2-99e0-b2f792cfcfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:28.394681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 366/721] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:58:28.685037 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57c9f-f9ce-48e1-a26c-be3eac5bdce7 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:28.889025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57c9f-f9ce-48e1-a26c-be3eac5bdce7 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:28.959721 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232067d-33ab-44b4-80f3-4a820c693802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:28.960678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232067d-33ab-44b4-80f3-4a820c693802 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:28.961164 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232067d-33ab-44b4-80f3-4a820c693802 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:28.961722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232067d-33ab-44b4-80f3-4a820c693802 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:28.976729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:28.983247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232067d-33ab-44b4-80f3-4a820c693802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5232067d-33ab-44b4-80f3-4a820c693802 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:28.988184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:28.989953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:28.992266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 356/722] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:28.996451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85e051-a2ee-49f2-99e0-b2f792cfcfda None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:29.007065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53281a-e91b-4da1-a8aa-41e0962012d7 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:29.007461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53281a-e91b-4da1-a8aa-41e0962012d7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:29.007694 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53281a-e91b-4da1-a8aa-41e0962012d7 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:29.008971 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 357/723] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:29.023522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de0adf-b4f8-4c9a-ac99-2838703c6f2d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:29.023976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de0adf-b4f8-4c9a-ac99-2838703c6f2d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:29.024225 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de0adf-b4f8-4c9a-ac99-2838703c6f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:29.062618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-62de0adf-b4f8-4c9a-ac99-2838703c6f2d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:29.063838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-62de0adf-b4f8-4c9a-ac99-2838703c6f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:29.079228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:29.247469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85e051-a2ee-49f2-99e0-b2f792cfcfda None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:29.268746 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 358/724] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 11:58:29.325261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:29.327311 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:29.328093 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:29.328879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:29.330519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:29.331363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:29.334433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:29.351606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 367/725] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:29.358880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de0adf-b4f8-4c9a-ac99-2838703c6f2d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:29.536215 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de0adf-b4f8-4c9a-ac99-2838703c6f2d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:29.579887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:29.588770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:29.589499 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:29.594060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:29.594060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:29.611046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:29.612857 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:29.634669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:29.635706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:29.636620 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:29.636620 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:29.636620 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:29.636620 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 359/726] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:29.643097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:29.663890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:29.784967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc9358-fc0d-40e1-af0e-550c15c66904 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:29.823486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:29.824470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:29.825092 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:29.825590 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:29.839172 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:29.845767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:29.846736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:29.847579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:29.850398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 360/727] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:29.856575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:29.920984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:29.925004 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:29.947396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bfb1d-5361-4681-b5a2-b4af4910d184 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:30.049979 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:30.051041 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:30.051539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:30.052566 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:30.054012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:30.054812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:30.060208 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:30.085068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcadf1-daf9-4e6a-9a80-96b8ec5af371 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:30.090612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 368/728] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:30.103382 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:30.159355 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:30.160260 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:30.160678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:30.161102 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:30.161944 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:30.162393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:30.164005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:30.174354 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 361/729] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:30.180194 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:30.364729 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d1b47-063e-4868-8d6e-505d7db75b7e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:30.372413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18c2e3-7235-4e92-b49b-69b2d0464c6c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:30.377574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:30.450480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:30.451854 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:30.457274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:30.457274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:30.475212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:30.476211 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:30.476668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:30.476788 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:30.477228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:30.485633 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:30.505632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:30.508426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:30.528618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:30.529704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:30.531156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:30.531156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:30.531156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:30.531803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 362/730] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:30.538450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:30.547572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:30.549423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:30.551375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 369/731] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:30.555140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:30.595928 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:30.781984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf5b73-5685-4bd2-b7f9-5b53c0b95072 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:30.802441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c55f68-0753-4018-8de9-9f4c5c3bfc71 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:30.866563 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:30.867562 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:30.868919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:30.869844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:30.871190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:30.872166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:30.874839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:30.878170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7378e-02ed-42ef-b274-c8ce3e5d22e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:30.879062 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7378e-02ed-42ef-b274-c8ce3e5d22e8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:30.879537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7378e-02ed-42ef-b274-c8ce3e5d22e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:30.880005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7378e-02ed-42ef-b274-c8ce3e5d22e8 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:30.880979 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7378e-02ed-42ef-b274-c8ce3e5d22e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:30.881494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7378e-02ed-42ef-b274-c8ce3e5d22e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:30.884175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7378e-02ed-42ef-b274-c8ce3e5d22e8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:30.915509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 363/732] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:30.919090 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:30.921764 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 370/733] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Sep 1 11:58:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:30.923461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d177f-2385-4c1a-a6e1-a887dbc33616 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:30.923985 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d177f-2385-4c1a-a6e1-a887dbc33616 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:30.924458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d177f-2385-4c1a-a6e1-a887dbc33616 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:30.926523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 371/734] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:30.927556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d177f-2385-4c1a-a6e1-a887dbc33616 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:30.959539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:31.012942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:31.060721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d177f-2385-4c1a-a6e1-a887dbc33616 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:31.105762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0d319-d488-4cb6-9865-91eaf1232b7c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:31.114588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c962e-42bd-4098-b8be-0ccf613e41e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:31.115321 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c962e-42bd-4098-b8be-0ccf613e41e5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.115610 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c962e-42bd-4098-b8be-0ccf613e41e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.115884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c962e-42bd-4098-b8be-0ccf613e41e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.148968 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d862a-b8fa-4a09-9a3d-ae454abc1f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:31.149628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d862a-b8fa-4a09-9a3d-ae454abc1f8f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.149936 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d862a-b8fa-4a09-9a3d-ae454abc1f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.150221 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d862a-b8fa-4a09-9a3d-ae454abc1f8f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.164711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d862a-b8fa-4a09-9a3d-ae454abc1f8f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:31.166235 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d862a-b8fa-4a09-9a3d-ae454abc1f8f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d77d862a-b8fa-4a09-9a3d-ae454abc1f8f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:31.192120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:31.194413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 364/735] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:58:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:31.196367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e23f4-75c1-4cd7-a831-63eb1f576ea8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.200602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e23f4-75c1-4cd7-a831-63eb1f576ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.200602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e23f4-75c1-4cd7-a831-63eb1f576ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.218222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c962e-42bd-4098-b8be-0ccf613e41e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:31.220453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c962e-42bd-4098-b8be-0ccf613e41e5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:31.227546 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d90e23f4-75c1-4cd7-a831-63eb1f576ea8 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:31.229570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d90e23f4-75c1-4cd7-a831-63eb1f576ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:31.326079 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 365/736] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:31.327954 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf8078-f7dd-4b48-a0c5-dc1a0e4f5424 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.328323 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf8078-f7dd-4b48-a0c5-dc1a0e4f5424 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.328603 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf8078-f7dd-4b48-a0c5-dc1a0e4f5424 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.336404 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 366/737] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:31.336404 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf8078-f7dd-4b48-a0c5-dc1a0e4f5424 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:31.342128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:31.353697 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:31.405838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 372/738] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 11:58:31.409296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fbdde-cc4c-4197-b26a-c35d363b0daf None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.410130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fbdde-cc4c-4197-b26a-c35d363b0daf None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.410629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fbdde-cc4c-4197-b26a-c35d363b0daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.443846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-9d0fbdde-cc4c-4197-b26a-c35d363b0daf None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:31.446407 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9d0fbdde-cc4c-4197-b26a-c35d363b0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:31.491591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf8078-f7dd-4b48-a0c5-dc1a0e4f5424 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:31.546110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 373/739] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:31.546110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911b96b-4d3b-40bb-9014-9e724da28880 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.546110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911b96b-4d3b-40bb-9014-9e724da28880 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.546110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911b96b-4d3b-40bb-9014-9e724da28880 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.546110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 374/740] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:31.546110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911b96b-4d3b-40bb-9014-9e724da28880 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:31.573408 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1206b2e-c520-4f86-8754-e57b2861ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:31.575597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1206b2e-c520-4f86-8754-e57b2861ab9b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.576520 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1206b2e-c520-4f86-8754-e57b2861ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.577492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1206b2e-c520-4f86-8754-e57b2861ab9b None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.579974 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1206b2e-c520-4f86-8754-e57b2861ab9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:31.580914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1206b2e-c520-4f86-8754-e57b2861ab9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:31.585105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1206b2e-c520-4f86-8754-e57b2861ab9b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:31.615762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 367/741] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:58:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:31.617446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c018c-e4d5-4e4e-a8cc-90e1298fcae3 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.617772 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c018c-e4d5-4e4e-a8cc-90e1298fcae3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.618051 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c018c-e4d5-4e4e-a8cc-90e1298fcae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.638067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b37c018c-e4d5-4e4e-a8cc-90e1298fcae3 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:31.640201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b37c018c-e4d5-4e4e-a8cc-90e1298fcae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:31.713896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:31.715225 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:31.720715 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911b96b-4d3b-40bb-9014-9e724da28880 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:31.737631 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 368/742] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 11:58:31.737631 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c018c-e4d5-4e4e-a8cc-90e1298fcae3 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:31.790484 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:31.791569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.792067 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.792554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.806282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:31.808291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:31.854122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:31.855143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:31.856058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:31.856058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:31.856058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:31.856058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 375/743] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:31.856805 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:31.896357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c018c-e4d5-4e4e-a8cc-90e1298fcae3 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:31.961419 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:31.962629 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:31.963101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:31.963558 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:31.982565 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:31.985932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:31.986661 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c146ce5-8455-4fe2-9baa-c5db78ede718 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:32.013575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:32.015328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:32.016290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:32.016290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:32.016290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:32.016290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 369/744] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:32.018146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:32.032069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:32.032704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:32.032988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:32.033264 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:32.033990 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:32.035377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:32.113872 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:32.159362 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 376/745] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:58:31 2022] POST /identity/v3/services => generated 251 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:32.160843 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:32.177803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:32.192786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666fffc-8687-4779-b64e-7520afcc5914 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.242426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:32.243099 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:32.243384 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:32.243660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:32.258587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:32.264851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:32.266726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 370/746] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:32.268306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:32.308666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c88c6-709a-465b-bb55-3b8f73de9860 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.359798 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:32.360434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:32.360716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:32.360993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:32.361684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:32.361997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:32.363418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:32.379958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 377/747] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:32.380876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:32.418530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:32.425544 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841375-a2d1-4c22-8071-dff223fba508 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.487217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:32.488095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:32.488616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:32.489112 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:32.490371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:32.490933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:32.493623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:32.496277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:32.519171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 371/748] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:32.519171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:32.566418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa0fe4-7a31-4313-a622-d7ec16a56387 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.616062 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:32.616842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:32.617131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:32.617488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:32.618571 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:32.620353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:32.674641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f98b0-47e0-49c9-970b-fcb51e15de53 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.722233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 378/749] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:58:32 2022] POST /identity/v3/services => generated 240 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:32.723339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:32.750819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:32.752219 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:32.752656 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:32.753078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:32.754007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:32.754613 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:32.757315 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:32.771679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 372/750] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:32.772978 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:32.808837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:32.897888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12077f-e2cf-4a6e-b4d7-b143ea7cc7fc None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.949688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:32.958921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:32.960453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:32.960453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:32.960453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:32.968299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022936b7-4fdd-4e11-b13f-18c627b77dfd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:32.974605 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:32.976087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:32.992286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:32.996023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 379/751] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:32.997812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:33.056482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:33.057896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:33.058346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:33.058792 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:33.076333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:33.080607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:33.097853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:33.100925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 373/752] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:33.104080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:33.143707 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694496c6-f68e-45c7-add6-dd2db0ce4ca8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:33.183103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:33.183779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:33.184045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:33.184318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:33.197468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:33.198969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:33.215908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:33.219992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:33.223470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:33.223470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:33.223470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:33.223470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 380/753] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:33.223470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:33.255708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:33.275104 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:33.302831 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2f389-b6b8-42c1-9156-5828715374db None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:33.343454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3d3d4-58a1-4e13-b7f4-4bed2a5dab4e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:33.382471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:33.383875 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:33.384222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:33.384844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:33.385122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:33.385392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:33.385727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:33.386140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:33.386446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:33.386943 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:33.387870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:33.388708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:33.389689 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:33.391470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:33.413030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 381/755] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:33.413283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 374/755] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:33.421637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:33.425866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:33.604638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b3e57-dbbc-48e0-8e70-647b45dcfe21 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:33.624880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4cd11-3fec-4553-84aa-c4c90b1c4849 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:33.628682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:33.667571 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:33.668774 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:33.669363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:33.669959 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:33.671825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:33.672680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:33.675293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:33.688659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:33.694947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:33.696240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:33.698597 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:33.698597 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:33.698882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 375/756] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:33.699445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=60c4eccc7fb0434e9251fa1b9bd7aec2) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:33.700048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:33.700461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:33.703471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:33.715177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 382/757] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:33 2022] GET /identity/v3/role_assignments?user.id=60c4eccc7fb0434e9251fa1b9bd7aec2&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:33.718892 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:33.927507 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efe723-0bc2-407d-841e-44516954a12a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:33.996130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:33.997640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:33.998351 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:33.999676 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:34.013072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:34.016160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:34.033853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:34.035713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:34.037637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:34.041947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 383/758] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 11:58:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:34.051960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c387854-9e2d-43ea-9cf4-dbba61504026 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:34.055181 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:34.132113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:34.133243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:34.133733 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:34.134134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:34.134310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/0c8225879a66415c8a1dc28584e4c626/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:34.145315 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=0c8225879a66415c8a1dc28584e4c626, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:34.148038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:34.237041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 376/759] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:33 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/0c8225879a66415c8a1dc28584e4c626/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:58:34.308863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbe7d0-d255-41f5-bdf0-cd4f77ef63fb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:34.406874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d970f6-8b6c-4068-aec3-d7d841be9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:34.407845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d970f6-8b6c-4068-aec3-d7d841be9572 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:34.408304 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d970f6-8b6c-4068-aec3-d7d841be9572 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:34.408757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d970f6-8b6c-4068-aec3-d7d841be9572 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:34.410050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d970f6-8b6c-4068-aec3-d7d841be9572 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:34.410585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d970f6-8b6c-4068-aec3-d7d841be9572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:34.413239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d970f6-8b6c-4068-aec3-d7d841be9572 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:34.467704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 384/760] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Sep 1 11:58:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:34.476802 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:34.599381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:34.677727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bda398-23fe-4fcd-bcb9-110db3be1325 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:34.760161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:34.761733 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:34.762495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:34.763238 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:34.912973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:34.916206 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:35.042572 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:35.146413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240becf1-0eed-43be-9c77-ffd821358515 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:35.147203 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240becf1-0eed-43be-9c77-ffd821358515 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:35.147388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240becf1-0eed-43be-9c77-ffd821358515 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:35.149789 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 385/761] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:58:35.158323 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528761-b33d-4c4e-90cd-6fc7edc4083f None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:35.158787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528761-b33d-4c4e-90cd-6fc7edc4083f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:35.159273 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528761-b33d-4c4e-90cd-6fc7edc4083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:35.169739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 377/762] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:58:35.175821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:35.215281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-8e528761-b33d-4c4e-90cd-6fc7edc4083f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:35.217359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-8e528761-b33d-4c4e-90cd-6fc7edc4083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:35.471701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a9f1-600b-4546-a887-5753804b2d80 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:35.479866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 386/763] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:35.546169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:35.547078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:35.547509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:35.547956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:35.559919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:35.562270 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:35.579538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:35.580718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:35.581795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:35.581795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:35.581795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:35.583607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 378/764] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:35.592446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528761-b33d-4c4e-90cd-6fc7edc4083f None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:35.642421 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:35.832560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528761-b33d-4c4e-90cd-6fc7edc4083f None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:35.848565 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:35.888727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:35.927194 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:35.928249 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:35.928634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:35.929126 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:35.930311 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:35.930851 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:35.933489 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:35.961867 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 387/765] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:35.970984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:36.125980 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c07d8-fb06-419a-8418-43418b8312ac None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:36.151819 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:36.196223 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a518c-cabb-4bfa-857b-58f3aa72c98b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:36.261638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:36.261638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.261638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.261638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.262815 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:36.262815 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:36.276317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e19b2-b3bf-4cff-88bd-0b19deea3aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:36.277900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e19b2-b3bf-4cff-88bd-0b19deea3aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.278579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e19b2-b3bf-4cff-88bd-0b19deea3aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.279356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e19b2-b3bf-4cff-88bd-0b19deea3aa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:36.280759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:36.282830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:36.284512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 379/766] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Sep 1 11:58:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:36.289443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:36.306431 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e19b2-b3bf-4cff-88bd-0b19deea3aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:36.313736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e19b2-b3bf-4cff-88bd-0b19deea3aa4 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:36.439111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-477e19b2-b3bf-4cff-88bd-0b19deea3aa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:36.467539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:36.468530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:36.469437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 388/767] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:36.470898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7842621-c9e7-4849-bdbf-912862cbc5bd None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.470898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7842621-c9e7-4849-bdbf-912862cbc5bd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.470898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7842621-c9e7-4849-bdbf-912862cbc5bd None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.472570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 389/768] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:36.474663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722a7d4-1886-4883-981f-4621cc3eb147 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:36.483666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc2343-5fdf-4293-867d-93498ff28a8b None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.484090 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc2343-5fdf-4293-867d-93498ff28a8b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.484469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc2343-5fdf-4293-867d-93498ff28a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.505022 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-dacc2343-5fdf-4293-867d-93498ff28a8b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:36.506299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-dacc2343-5fdf-4293-867d-93498ff28a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:36.521653 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:36.522756 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.523362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.523866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.524742 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:36.525268 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:36.527292 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:36.534207 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 380/769] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Sep 1 11:58:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:36.540093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:36.587404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 390/770] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:58:36.588312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc2343-5fdf-4293-867d-93498ff28a8b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:36.691489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7a38-dfca-4531-a791-1154a2952b49 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:36.694426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:36.737282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:36.738491 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.739569 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.740289 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.741258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:36.741806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:36.743729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:36.753162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:36.760853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 381/771] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:36.769380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:36.800035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc2343-5fdf-4293-867d-93498ff28a8b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:36.910743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9895ce-6cdb-404a-a379-3caf178e9dab None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:36.915659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:36.916324 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.916606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.916877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.917554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:36.917902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:36.919334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:36.940456 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 391/772] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Sep 1 11:58:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:36.949309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:36.969258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ec6d0-c2a0-434b-9e49-f53720711732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:36.970311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ec6d0-c2a0-434b-9e49-f53720711732 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:36.970811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ec6d0-c2a0-434b-9e49-f53720711732 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:36.972083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ec6d0-c2a0-434b-9e49-f53720711732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:36.973221 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ec6d0-c2a0-434b-9e49-f53720711732 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:36.973806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ec6d0-c2a0-434b-9e49-f53720711732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:36.975599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ec6d0-c2a0-434b-9e49-f53720711732 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:36.983777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 382/773] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:36 2022] GET /identity/v3/role_assignments?user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:37.070321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fee5a-1082-4bdd-8aa0-95e7723bcae3 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:37.070972 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fee5a-1082-4bdd-8aa0-95e7723bcae3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:37.071483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fee5a-1082-4bdd-8aa0-95e7723bcae3 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:37.073097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 383/774] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:37.082160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b1368-3f7b-4937-bbf1-4d03179d3425 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:37.082651 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b1368-3f7b-4937-bbf1-4d03179d3425 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:37.083424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b1368-3f7b-4937-bbf1-4d03179d3425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:37.109118 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad0717-fef9-413c-9b73-6d45be495327 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:37.144095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f91b1368-3f7b-4937-bbf1-4d03179d3425 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:37.146751 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:37.154707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f91b1368-3f7b-4937-bbf1-4d03179d3425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:37.159815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:37.161468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:37.162017 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:37.162476 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:37.163408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] RBAC: Adding query filter params (interface=public, service_id=0fb88220a6c944cb9a6f8e039fc205e2, region_id=RegionOne) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:37.163932 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:37.165585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:37.179206 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 392/775] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 11:58:36 2022] GET /identity/v3/endpoints?service_id=0fb88220a6c944cb9a6f8e039fc205e2&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:37.185141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:37.296017 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:37.314211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 384/776] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:37.315883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b1368-3f7b-4937-bbf1-4d03179d3425 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:37.333416 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96ffe6-0818-485f-9e0d-63c66305f154 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:37.392728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:37.394209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:37.394843 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:37.395408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:37.396609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:37.399317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:37.410876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 393/777] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:58:37 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:37.494159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b1368-3f7b-4937-bbf1-4d03179d3425 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:37.554313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:37.555216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:37.555646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:37.556064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] PATH_INFO: `/v3/services/network` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:37.556805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:37.558398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:37.572440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:37.573692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:37.574664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:37.577578 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 385/778] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:37 2022] GET /identity/v3/services/network => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:37.587811 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:37.750445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:37.837871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8854a-5127-49f4-89b4-1eceb57a7db5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:37.879427 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:37.935611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:37.937250 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:37.937771 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:37.938266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:37.939112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] RBAC: Adding query filter params (name=network) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:37.939622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:37.941247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:37.959922 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 394/779] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:58:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:37.968994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:38.036127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:38.166492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f171d-f5ac-459b-8e6e-f23f6d2d18af None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:38.247041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:38.247715 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:38.248002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:38.248281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:38.250687 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ad49a-2f93-42e9-b54e-a66262018b82 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:38.278610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:38.284473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:38.286333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:38.288038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:38.293140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 386/780] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:38.299764 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:38.306370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:38.345029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:38.347926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:38.348521 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:38.354124 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:38.354835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] RBAC: Adding query filter params (type=network) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:38.355149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:38.356555 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:38.372460 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 395/781] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:58:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:38.387005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:38.464341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c8c9e-095c-48e6-954c-811a6a913fc0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:38.494446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:38.527311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521441-dbf1-498e-b1a9-60c71eb657de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:38.528595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521441-dbf1-498e-b1a9-60c71eb657de None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:38.529110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521441-dbf1-498e-b1a9-60c71eb657de None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:38.529579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521441-dbf1-498e-b1a9-60c71eb657de None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:38.530890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521441-dbf1-498e-b1a9-60c71eb657de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:38.531853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521441-dbf1-498e-b1a9-60c71eb657de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:38.535138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521441-dbf1-498e-b1a9-60c71eb657de None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:38.552015 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 387/782] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:38.554506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679299b-8e7e-4bba-ab86-c8f00d143d3c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:38.555095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679299b-8e7e-4bba-ab86-c8f00d143d3c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:38.555663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679299b-8e7e-4bba-ab86-c8f00d143d3c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:38.565116 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 388/783] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:38.565116 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679299b-8e7e-4bba-ab86-c8f00d143d3c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:38.645536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eef2e4-f3f5-4d3b-bdef-3183283ce49f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:38.704298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53adabe-c89b-40a0-937f-c919eb97ce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:38.705668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53adabe-c89b-40a0-937f-c919eb97ce71 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:38.706370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53adabe-c89b-40a0-937f-c919eb97ce71 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:38.707073 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53adabe-c89b-40a0-937f-c919eb97ce71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:38.708533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53adabe-c89b-40a0-937f-c919eb97ce71 None admin] RBAC: Adding query filter params (interface=public, service_id=379deb6626764f408944c39dc9edd9f0, region_id=RegionOne) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:38.709429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53adabe-c89b-40a0-937f-c919eb97ce71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:38.712195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53adabe-c89b-40a0-937f-c919eb97ce71 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:38.723818 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 396/784] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 11:58:38 2022] GET /identity/v3/endpoints?service_id=379deb6626764f408944c39dc9edd9f0&interface=public®ion_id=RegionOne => generated 194 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:38.726394 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c53bd-019d-4819-8323-06221ce4de15 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:38.727175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c53bd-019d-4819-8323-06221ce4de15 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:38.727955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c53bd-019d-4819-8323-06221ce4de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:38.759847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-d46c53bd-019d-4819-8323-06221ce4de15 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:38.761765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679299b-8e7e-4bba-ab86-c8f00d143d3c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:38.762593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d46c53bd-019d-4819-8323-06221ce4de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:38.825423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:38.825769 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:38.826115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:38.826536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:38.842029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:38.849182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:38.858459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:38.881033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:38.882912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:38.884670 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:38.884670 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:38.884670 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:38.884670 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 389/785] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:38.885920 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:38.897698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 397/786] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:38.899578 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c53bd-019d-4819-8323-06221ce4de15 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:38.946909 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:39.085855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc97f4-256c-428b-8b23-572ec4cf1e9e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:39.152223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:39.153470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:39.154061 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:39.154693 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:39.156369 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:39.159046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:39.169657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 390/787] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:58:38 2022] GET /identity/v3/services => generated 772 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:39.219122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c53bd-019d-4819-8323-06221ce4de15 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:39.277989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274a5a9-141e-4305-85bf-3b1172451c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:39.278845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274a5a9-141e-4305-85bf-3b1172451c8a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:39.279007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274a5a9-141e-4305-85bf-3b1172451c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:39.279235 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274a5a9-141e-4305-85bf-3b1172451c8a None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:39.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274a5a9-141e-4305-85bf-3b1172451c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:39.280675 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274a5a9-141e-4305-85bf-3b1172451c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:39.282131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274a5a9-141e-4305-85bf-3b1172451c8a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:39.307731 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 398/788] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:39.314798 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:39.452867 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1b536-979f-4c6c-a3b1-a57dac7b4418 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:39.453164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1b536-979f-4c6c-a3b1-a57dac7b4418 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:39.453419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1b536-979f-4c6c-a3b1-a57dac7b4418 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:39.455124 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 399/789] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:39.460849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac17b59-b0aa-476e-b9c0-deeda91ff9f3 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:39.461140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac17b59-b0aa-476e-b9c0-deeda91ff9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:39.461403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac17b59-b0aa-476e-b9c0-deeda91ff9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:39.507653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-fac17b59-b0aa-476e-b9c0-deeda91ff9f3 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:39.510308 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:39.512420 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-fac17b59-b0aa-476e-b9c0-deeda91ff9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:39.521362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9e97f-2cc3-473a-b996-f1734bfc8fe1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:39.574660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:39.586089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:39.587214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:39.587676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:39.588135 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:39.606301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:39.608783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:39.625839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 400/790] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:58:39.631291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac17b59-b0aa-476e-b9c0-deeda91ff9f3 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:39.640735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:39.642125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:39.643175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:39.655844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 391/791] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:39.665837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:39.858143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac17b59-b0aa-476e-b9c0-deeda91ff9f3 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:39.927924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:39.929102 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:39.929556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:39.930022 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:39.945488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:39.948206 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18358f-b269-4220-bc38-348b9fc86e2f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:39.950768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:39.952287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:39.953129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:39.962378 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 401/792] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:39.962378 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.004829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:40.020537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.021161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.021506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.021918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.022570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:40.022878 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:40.024444 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:40.057696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 392/793] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Sep 1 11:58:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:40.058739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.081464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:40.170757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b55a4-a67d-484d-85de-a904c5e93a7e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:40.228284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.229163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.229551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.229967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.243189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deaf8de-336d-45c3-9af8-317898fc5117 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:40.247094 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:40.257009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:40.258161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:40.259030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:40.259685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 402/794] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:40.260891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.293841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e9401-57b4-4848-9c88-2b3babaa95d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.294969 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e9401-57b4-4848-9c88-2b3babaa95d5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.296037 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e9401-57b4-4848-9c88-2b3babaa95d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.296584 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e9401-57b4-4848-9c88-2b3babaa95d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.297431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e9401-57b4-4848-9c88-2b3babaa95d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:40.297918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e9401-57b4-4848-9c88-2b3babaa95d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:40.299461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e9401-57b4-4848-9c88-2b3babaa95d5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:40.310011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 393/795] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:40.311892 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66c3a-4ff6-4232-b15d-5bb3ece6db06 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.312177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66c3a-4ff6-4232-b15d-5bb3ece6db06 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.312452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66c3a-4ff6-4232-b15d-5bb3ece6db06 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.314023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 394/796] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:40.314517 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66c3a-4ff6-4232-b15d-5bb3ece6db06 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.397005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:40.410357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea9199-f512-4c43-9760-8c3eaead91a9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:40.432890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:40.456386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.457035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.457317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.457590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.469036 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:40.470543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:40.474148 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66c3a-4ff6-4232-b15d-5bb3ece6db06 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:40.489422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:40.490984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:40.492749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:40.492749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:40.492749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:40.495471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 403/797] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:40.496411 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.538716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358f338-32c8-4959-84e7-25e9b3e74d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.540034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358f338-32c8-4959-84e7-25e9b3e74d7f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.540457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358f338-32c8-4959-84e7-25e9b3e74d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.540736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358f338-32c8-4959-84e7-25e9b3e74d7f None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.542043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358f338-32c8-4959-84e7-25e9b3e74d7f None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:40.542545 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358f338-32c8-4959-84e7-25e9b3e74d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:40.544874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358f338-32c8-4959-84e7-25e9b3e74d7f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:40.573450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 395/798] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:40.575645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceca76-99d2-4679-85cf-4664f10eeb8d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.576127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceca76-99d2-4679-85cf-4664f10eeb8d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.577108 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aceca76-99d2-4679-85cf-4664f10eeb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.610186 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-4aceca76-99d2-4679-85cf-4664f10eeb8d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:40.612198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-4aceca76-99d2-4679-85cf-4664f10eeb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:40.660588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f031a8-227c-4dd2-83d4-98a7cbf0c6e6 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:40.705182 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 396/799] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:40.706405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceca76-99d2-4679-85cf-4664f10eeb8d None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.730115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.731434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.731905 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.737276 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.744371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:40.747059 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:40.748965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:40.766804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:40.767820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:40.768794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:40.768794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:40.768794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:40.773155 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 404/800] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:40.774277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.799144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:40.852165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aceca76-99d2-4679-85cf-4664f10eeb8d None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:40.907711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.908671 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.909098 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.909523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:40.910699 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:40.911287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:40.913392 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:40.924352 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae827f0-a1c1-4ef0-a8f8-ffb3c58abf6c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:40.938818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 397/801] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:40.938818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:40.980645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:40.982082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:40.983765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:40.984301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.004268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.006292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:41.032109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:41.033119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:41.034064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:41.034064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:41.034064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:41.036535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 405/802] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:41.037881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:41.109794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ac07c-56ac-4ba0-8c52-a092eee19f0e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.122021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:41.147416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:41.164144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:41.165254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.165860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.166414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.167666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:41.168223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.171835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.205753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d2b4-ce3e-4e49-9e4a-d443092a4d14 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.206661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 398/803] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:41.209866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:41.252171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:41.252711 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.252989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.253268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.261225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.262745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:41.275727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:41.277486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:41.280369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:41.282060 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 406/804] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:41.283012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:41.380389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8e134-c74b-44d2-b851-d5162afdfb6a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.425298 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:41.426112 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.426512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.427426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.427966 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:41.428425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.430430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.452592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 399/805] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:41.454345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:41.478045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a522-6d22-4f10-a018-2a7179cb2f7a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.484227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:41.521041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:41.536858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:41.537935 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.538634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.539538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.549100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.550942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:41.566709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:41.568554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:58:41.571310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:41.574659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 407/806] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:41.576599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:41.630072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461708c-dfed-4d01-adb0-acf51bf3ecce None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.685496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:41.686747 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.687261 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.687765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.688863 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.691087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:41.708082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:41.716768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:41.719132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 400/807] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:41.720871 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:41.721081 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a227-aa50-4c5f-9178-6c9a4f93cc06 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.764358 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:41.765324 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.765875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.766317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.767508 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:41.768017 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.770519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.791890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 408/808] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:41.793067 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:41.833142 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:41.891426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951900c3-514b-4e78-b180-897ae54e16e1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.910409 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:41.941831 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6d943-ed62-4108-bb8a-468dc43d2771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:41.942636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6d943-ed62-4108-bb8a-468dc43d2771 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.943002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6d943-ed62-4108-bb8a-468dc43d2771 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.943390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6d943-ed62-4108-bb8a-468dc43d2771 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.952939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fed44-3f31-4817-b7a1-b3b80e4f09a7 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:41.953788 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6d943-ed62-4108-bb8a-468dc43d2771 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:41.955481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6d943-ed62-4108-bb8a-468dc43d2771 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0dd6d943-ed62-4108-bb8a-468dc43d2771 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:41.969585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:41.971913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 401/809] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25a619-bef6-47d9-957e-2944dabd6c4c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:41.974064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25a619-bef6-47d9-957e-2944dabd6c4c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:41.975897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25a619-bef6-47d9-957e-2944dabd6c4c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:41.976958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 402/810] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:41.977993 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25a619-bef6-47d9-957e-2944dabd6c4c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:42.009163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:42.010149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:42.010597 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:42.011038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:42.012261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:42.012759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:42.015889 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:42.039661 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 409/811] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:42.041515 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:42.117241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25a619-bef6-47d9-957e-2944dabd6c4c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:42.154738 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:42.155382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:42.155730 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:42.156003 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:42.156668 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:42.156975 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:42.158485 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:42.160332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:42.163939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 403/812] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Sep 1 11:58:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:42.164853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:42.182483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4888ef-9d37-4e57-8e6a-dc8cf2b0080f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:42.209511 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:42.234888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dcc3c-e06b-4551-8f56-beca6b133bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:42.235942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dcc3c-e06b-4551-8f56-beca6b133bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:42.236382 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dcc3c-e06b-4551-8f56-beca6b133bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:42.236791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dcc3c-e06b-4551-8f56-beca6b133bc6 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/60c4eccc7fb0434e9251fa1b9bd7aec2/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:42.240303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dcc3c-e06b-4551-8f56-beca6b133bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=60c4eccc7fb0434e9251fa1b9bd7aec2, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:42.242930 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dcc3c-e06b-4551-8f56-beca6b133bc6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:42.301480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b010a-95c6-4b79-b850-83c5ca6474a4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:42.343158 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:42.343790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:42.344069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:42.344341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:42.345091 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:42.345397 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:42.346929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:42.367588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 410/813] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:42 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/60c4eccc7fb0434e9251fa1b9bd7aec2/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:58:42.369990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 404/814] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Sep 1 11:58:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:42.371156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:42.371349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930e13d-ba23-4133-93d6-ac43d5a3df82 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:42.378445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930e13d-ba23-4133-93d6-ac43d5a3df82 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:42.378445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930e13d-ba23-4133-93d6-ac43d5a3df82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:42.402671 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-5930e13d-ba23-4133-93d6-ac43d5a3df82 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:42.404534 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-5930e13d-ba23-4133-93d6-ac43d5a3df82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:42.533450 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 411/815] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:42.535833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930e13d-ba23-4133-93d6-ac43d5a3df82 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:42.543957 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:42.563989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43015cca-9c8d-47f3-b7b5-87ce899361f4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:42.630758 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:42.632899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:42.632899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:42.633236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:42.693590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:42.761923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930e13d-ba23-4133-93d6-ac43d5a3df82 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:42.797263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:42.800837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:42.830614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:42.831817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:42.832300 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:42.832788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:42.834046 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:42.835193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:42.837390 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:42.847756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 412/816] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Sep 1 11:58:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:42.849484 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:42.945088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:42.958370 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 405/817] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:58:42.960082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:43.040880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520be0ef-9c65-46b7-bce1-247d88ce07d4 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.084411 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:43.085447 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:43.086076 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:43.086806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:43.091918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:43.102258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:43.104162 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:43.124626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:43.126788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:43.128728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:43.128728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:43.128728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:43.131288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 413/818] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:43.132051 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:43.148041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a678c48-959d-420d-bc8b-0a37e8d9a88e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.202199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:43.202967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:43.203424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:43.203824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] PATH_INFO: `/v3/services/network` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:43.204549 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:43.206823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:43.218304 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:43.227011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:43.227876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 406/819] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:42 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:43.228709 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:43.294229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94ebb-e318-4764-a091-1ee26c69a9fe None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.296642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:43.346022 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:43.347239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:43.347799 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:43.348277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:43.349403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:43.352005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:43.364948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce834-b519-4b1d-b459-5bf92f3f3271 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.406136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:43.407199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:43.407674 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:43.408153 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:43.426558 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:43.428298 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:43.446506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:43.448381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:43.450211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:43.450211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:43.450211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:43.454842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 407/820] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:43.455450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:43.477701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 414/821] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:58:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:43.479403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:43.581327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:43.636140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6b178-9384-41a1-9322-6922d3f2173a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.662038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:43.692573 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:43.693348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:43.693766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:43.694386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:43.695142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:43.695554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:43.697309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:43.706977 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fc0f9-34a3-4c40-b3c7-c9eb582ab499 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.713443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 408/822] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:43.714507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:43.772513 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:43.773585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:43.774106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:43.774598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:43.775754 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] RBAC: Adding query filter params (name=network) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:43.776322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:43.779117 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:43.788626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 415/823] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:58:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:43.791400 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:43.917006 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83034b-a92f-45d6-8ee5-ebb9cd010ef1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.987517 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:43.988589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:43.988700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972e23c-1377-430b-8123-5de223cdb761 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:43.989087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:43.989587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:43.990824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:43.991567 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:43.993110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:43.995514 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.020742 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 409/824] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:44.022322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:44.051330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:44.051989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.052273 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.052549 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.062188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:44.064058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:44.080282 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:44.082144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:44.083945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:44.087161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 416/825] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:44.088543 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:44.089726 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:44.222202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adbbdb-d66b-405f-ad2c-1c4c53b5d04f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:44.275674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:44.276383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.276751 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.277092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.285796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:44.287397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:44.303779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:44.304756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:44.305685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 417/826] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:44.306869 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:44.344392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:44.370471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6493e2-e1d3-44a9-b59b-eec61967a4f8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:44.410204 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:44.411249 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.411737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.412229 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.413198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:44.413817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:44.415678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.422829 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 410/827] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:58:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:44.424228 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:44.505955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f8e49-21bf-449c-beea-9dc21de6dbdb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:44.534209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:44.548952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56077323-179e-412b-87e8-acc7fc4760e1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:44.578405 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe16d3-12de-41f6-8eff-09f6c3a4143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:44.580227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe16d3-12de-41f6-8eff-09f6c3a4143e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.580989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe16d3-12de-41f6-8eff-09f6c3a4143e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.581781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe16d3-12de-41f6-8eff-09f6c3a4143e None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.584216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe16d3-12de-41f6-8eff-09f6c3a4143e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:44.585029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe16d3-12de-41f6-8eff-09f6c3a4143e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:44.588146 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe16d3-12de-41f6-8eff-09f6c3a4143e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.591659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:44.592329 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.592603 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.592879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.593587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:44.593924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:44.595586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.610389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 411/828] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:44.611316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:44.614895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 418/829] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:44.617051 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95db51-6966-4447-807c-668c4fadeb2e None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.617639 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95db51-6966-4447-807c-668c4fadeb2e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.618366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95db51-6966-4447-807c-668c4fadeb2e None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.622999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 419/830] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:44.625380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95db51-6966-4447-807c-668c4fadeb2e None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:44.733165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596e701-ce81-43c0-95c1-6f4fd63473af None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:44.734573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:44.755247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95db51-6966-4447-807c-668c4fadeb2e None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:44.782073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:44.782996 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.783464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.783908 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.784639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:44.786006 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.807742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592732e9-8900-4d36-b63f-82984621e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:44.808508 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592732e9-8900-4d36-b63f-82984621e7e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.808859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592732e9-8900-4d36-b63f-82984621e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.809207 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592732e9-8900-4d36-b63f-82984621e7e3 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/4e0eeaf94d9749c7b0dc15a97a3bf81a/roles/bc60629774714a078250dbc704217d2a` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:44.826798 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592732e9-8900-4d36-b63f-82984621e7e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=4e0eeaf94d9749c7b0dc15a97a3bf81a, role_id=bc60629774714a078250dbc704217d2a)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:44.828359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592732e9-8900-4d36-b63f-82984621e7e3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:44.929496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 412/831] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:58:44 2022] POST /identity/v3/endpoints => generated 337 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:44.934128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:44.945247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:44.950216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 420/832] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:58:44 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/4e0eeaf94d9749c7b0dc15a97a3bf81a/roles/bc60629774714a078250dbc704217d2a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:58:44.952135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2388bc7-b215-4b88-9501-1424d1da076b None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:44.952632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2388bc7-b215-4b88-9501-1424d1da076b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:44.953160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2388bc7-b215-4b88-9501-1424d1da076b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:45.016182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-c2388bc7-b215-4b88-9501-1424d1da076b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:45.018261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-c2388bc7-b215-4b88-9501-1424d1da076b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:45.203600 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 421/833] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:45.242751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443cb10-4898-4f80-b9f6-e4f504723cad None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:45.282933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:45.283503 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:45.283780 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:45.284046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:45.285310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=0c8225879a66415c8a1dc28584e4c626) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:45.285310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:45.288091 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:45.295742 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 413/834] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:44 2022] GET /identity/v3/role_assignments?user.id=0c8225879a66415c8a1dc28584e4c626&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:45.504087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2388bc7-b215-4b88-9501-1424d1da076b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:45.590258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:45.640954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2388bc7-b215-4b88-9501-1424d1da076b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:45.683015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3317f-f340-447f-8388-9edfab0615d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:45.683658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3317f-f340-447f-8388-9edfab0615d0 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:45.683952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3317f-f340-447f-8388-9edfab0615d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:45.684202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3317f-f340-447f-8388-9edfab0615d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:45.698380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3317f-f340-447f-8388-9edfab0615d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-88f3317f-f340-447f-8388-9edfab0615d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:45.703590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:45.705490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:45.707247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 422/835] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:45.715776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:45.799480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:45.813031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353ecc-7b50-4ae3-814e-61774a9ef2aa None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:45.813908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353ecc-7b50-4ae3-814e-61774a9ef2aa None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:45.814196 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2353ecc-7b50-4ae3-814e-61774a9ef2aa None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:45.816685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 423/836] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:58:45.823888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3388a9-e979-4f78-ae41-739d4cdc5b7f None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:45.824525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3388a9-e979-4f78-ae41-739d4cdc5b7f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:45.824861 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3388a9-e979-4f78-ae41-739d4cdc5b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:45.845054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef79ef2-a786-49fc-b1b7-dd36b73d81b1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:45.851518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-3c3388a9-e979-4f78-ae41-739d4cdc5b7f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:45.853418 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3c3388a9-e979-4f78-ae41-739d4cdc5b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:45.891339 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:45.892005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:45.893049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:45.893324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:45.894143 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:45.894461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:45.896271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:45.908345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 414/837] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:58:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:45.914077 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:45.987037 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 424/838] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:46.060597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f84c3e-5fee-46e5-b14b-8ae2d947f22b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:46.104689 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:46.114100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:46.119328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:46.119328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:46.119328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:46.138000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:46.141281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:46.172866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:46.174795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:46.176602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:46.176602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:46.176602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:46.180463 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 415/839] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:46.189709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3388a9-e979-4f78-ae41-739d4cdc5b7f None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:46.259254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:46.282931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:46.364359 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3388a9-e979-4f78-ae41-739d4cdc5b7f None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:46.396599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e0e12-1da5-4d02-8841-35e0362f70b0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:46.424091 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:46.424716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:46.425000 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:46.425271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:46.426156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:46.426470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:46.427861 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:46.451014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 425/840] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:46.452841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:46.455894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:46.455894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:46.456217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:46.456349 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:46.456956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:46.458436 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:46.469461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:46.471527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:46.473201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 416/841] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:46.479475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:46.595227 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:46.619746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67013c23-29aa-47f3-8c14-ac6f8aa321fb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:46.653770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:46.664280 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec53518-baf9-4021-9b4c-c0ad54da89b9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:46.694781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:46.695592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:46.695936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:46.696281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:46.705052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:46.706763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:46.720732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:46.722659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:46.724448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 426/842] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:58:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:46.729360 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:46.736851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:46.737637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:46.737993 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:46.738274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:46.738919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:46.739226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:46.741132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:46.752734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 417/843] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:46.756336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:46.863346 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea7105-9908-4a27-b71c-9ff58870dd6d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:46.938322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:46.939819 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:46.940486 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:46.941068 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:46.942054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:46.942552 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:46.945080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:46.957454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b6f2-1582-4139-b261-20015e8c475b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:46.975631 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 427/844] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Sep 1 11:58:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:46.980078 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:46.983928 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:47.010928 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:47.011853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.012292 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.012811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.013945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:47.014971 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:47.015436 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:47.018129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:47.024111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 418/845] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:47.106204 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e808e-e459-4be3-97e0-557bf3cbf075 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:47.149674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:47.150341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.150625 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.150906 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.239110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:47.240731 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:47.273649 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:47.384892 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 428/846] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:46 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:58:47.392330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:47.509536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31db08-e8b2-4ca4-9a11-89c85fce8b0e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:47.550120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e9f62-b86c-4c46-994e-7f333f4ec1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:47.551048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e9f62-b86c-4c46-994e-7f333f4ec1ef None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.551473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e9f62-b86c-4c46-994e-7f333f4ec1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.551806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e9f62-b86c-4c46-994e-7f333f4ec1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.564535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e9f62-b86c-4c46-994e-7f333f4ec1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:47.566031 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e9f62-b86c-4c46-994e-7f333f4ec1ef None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a43e9f62-b86c-4c46-994e-7f333f4ec1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:47.582314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:47.583335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:47.584243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:47.584243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:47.584243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:47.584243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 419/847] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:47.589465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:47.678042 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ee46f-755f-4a01-b27f-6abad1802c56 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.678349 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ee46f-755f-4a01-b27f-6abad1802c56 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.678621 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ee46f-755f-4a01-b27f-6abad1802c56 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.680300 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 420/848] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:58:47.686810 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32390661-f96e-4a81-9dc8-ce8c75a3d38c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.687232 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32390661-f96e-4a81-9dc8-ce8c75a3d38c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.687642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32390661-f96e-4a81-9dc8-ce8c75a3d38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.708118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-32390661-f96e-4a81-9dc8-ce8c75a3d38c None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:47.709685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-32390661-f96e-4a81-9dc8-ce8c75a3d38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:47.727327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e46d8-1dfc-4109-b1dc-08b4c0424f4a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:47.735218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:47.761499 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:47.771146 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca25c8-6c43-42f5-a32f-64d18dae4d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:47.772084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca25c8-6c43-42f5-a32f-64d18dae4d1c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.772549 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca25c8-6c43-42f5-a32f-64d18dae4d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.773032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca25c8-6c43-42f5-a32f-64d18dae4d1c None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.773976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca25c8-6c43-42f5-a32f-64d18dae4d1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:47.774494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca25c8-6c43-42f5-a32f-64d18dae4d1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:47.776152 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca25c8-6c43-42f5-a32f-64d18dae4d1c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:47.790419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 429/849] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:47.792184 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1c3a-9389-42f5-8794-0fb9cf6472a2 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.798841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1c3a-9389-42f5-8794-0fb9cf6472a2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.801271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 421/850] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:47.801403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1c3a-9389-42f5-8794-0fb9cf6472a2 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.801403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 430/851] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:47.801543 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32390661-f96e-4a81-9dc8-ce8c75a3d38c None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:47.801781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d96078-234e-45d4-85ed-f2c223953309 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.804298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d96078-234e-45d4-85ed-f2c223953309 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.804298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d96078-234e-45d4-85ed-f2c223953309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.824054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-d9d96078-234e-45d4-85ed-f2c223953309 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:47.825419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d9d96078-234e-45d4-85ed-f2c223953309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:47.914910 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 431/852] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:47.925753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32390661-f96e-4a81-9dc8-ce8c75a3d38c None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:47.964157 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:47.965639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:47.966095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:47.966522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:47.974820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:47.976664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:47.989636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:47.990874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:47.991976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:47.994192 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 422/853] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:48.002776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d96078-234e-45d4-85ed-f2c223953309 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.086861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.110275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:48.120334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d96078-234e-45d4-85ed-f2c223953309 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:48.144574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:48.175909 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:48.176538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:48.176825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:48.177102 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:48.177828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:48.178140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:48.179702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:48.194193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 432/854] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:48.204692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.248726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c10290-0a1c-4dbf-9b70-23029a3c879c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:48.301343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:48.302025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:48.302309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:48.302587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:48.317653 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:48.323855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:48.325745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 423/855] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:48.327538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.360629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99302-275e-455f-8f03-84792daa49d8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:48.406685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:48.407419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:48.407708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:48.407984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:48.408733 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=60c4eccc7fb0434e9251fa1b9bd7aec2) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:48.409044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:48.410533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:48.417182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 433/856] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/role_assignments?user.id=60c4eccc7fb0434e9251fa1b9bd7aec2&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:48.418072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.468256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:48.510586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebe6ab-0a0a-42db-8ef2-e869167c02b2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:48.527414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:48.578191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:48.579390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:48.579965 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:48.580483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:48.581697 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:48.583508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:48.600364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:48.602568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:58:48.604681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:48.613796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 424/857] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:48.616127 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.653326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979b920-52a7-48f3-977d-a2e154b89fad None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:48.716621 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:48.718171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:48.718630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:48.719079 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:48.720441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:48.720951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:48.723474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:48.737945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 434/858] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:48.744085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.786497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191ede3-212a-4917-848f-5e05e8004e26 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:48.836123 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:48.837891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:48.837891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:48.837891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:48.838976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:48.838976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:48.842667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:48.852095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 425/859] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:48.861853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:48.866709 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:48.941319 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4571eb0-53d9-444b-a71c-83d69825f4a2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:48.949257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:49.030857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:49.031991 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:49.032494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:49.033166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:49.037330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495a9d3-a065-42fe-9cf2-db2c4f68ad36 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:49.056166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:49.059397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:49.092825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:49.094682 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:49.096510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:49.096510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:49.096510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:49.097777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 435/860] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:49.109474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:49.119538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:49.120924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:49.121582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:49.122319 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:49.124378 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:49.125178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:49.127796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:49.141907 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 426/861] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:58:48 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:49.328045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834995b-1f58-4199-8165-e7b2c1c4ec82 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:49.392059 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:49.393637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:49.394219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:49.394787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:49.396063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:49.396684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:49.399377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:49.421404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 436/862] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:49.431200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:49.558121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:49.620910 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872abcbf-ece6-4aca-88f5-836860e101d2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:49.693517 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff706d2-389a-4883-aec9-9cc4c0c5d341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:49.694404 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff706d2-389a-4883-aec9-9cc4c0c5d341 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:49.694684 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff706d2-389a-4883-aec9-9cc4c0c5d341 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:49.695308 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff706d2-389a-4883-aec9-9cc4c0c5d341 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:49.704596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff706d2-389a-4883-aec9-9cc4c0c5d341 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:49.706264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff706d2-389a-4883-aec9-9cc4c0c5d341 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-cff706d2-389a-4883-aec9-9cc4c0c5d341 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:49.719288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:49.721383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:49.723396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:49.733657 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 427/863] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 11:58:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:49.735745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:49.809478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:49.811750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff51d5e-efd5-42df-b7f2-32ece4a109c8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:49.812053 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff51d5e-efd5-42df-b7f2-32ece4a109c8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:49.812345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff51d5e-efd5-42df-b7f2-32ece4a109c8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:49.824493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 428/864] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:49.827407 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afdaa-45f0-4dde-b354-ff5153f6292a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:49.837818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afdaa-45f0-4dde-b354-ff5153f6292a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:49.837818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afdaa-45f0-4dde-b354-ff5153f6292a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:49.869530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-686afdaa-45f0-4dde-b354-ff5153f6292a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:49.871230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-686afdaa-45f0-4dde-b354-ff5153f6292a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:49.950747 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 429/865] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:49.959481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5cd39-ea6a-4a14-b977-bd28133aee8e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:50.026028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:50.027071 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:50.027592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:50.027958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:50.029202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:50.030302 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:50.033150 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:50.061029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 437/866] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Sep 1 11:58:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:50.066458 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afdaa-45f0-4dde-b354-ff5153f6292a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:50.128509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:50.174173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afdaa-45f0-4dde-b354-ff5153f6292a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:50.212634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:50.213155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:50.213420 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:50.213729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:50.287544 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:50.289023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:50.360392 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:50.400371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 430/867] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 11:58:50.406585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:50.406941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:50.492683 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:50.522849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fe82b-2ea1-4608-b00b-a23848f041bd None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:50.563254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:50.564030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:50.564309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:50.564580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:50.565393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:50.566813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:50.576140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282fd2b-4881-478e-a08d-3fafc1725806 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:50.631017 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8bf34-4cba-423b-8438-2b075017e469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:50.631670 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8bf34-4cba-423b-8438-2b075017e469 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:50.631973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8bf34-4cba-423b-8438-2b075017e469 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:50.632247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8bf34-4cba-423b-8438-2b075017e469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:50.646282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8bf34-4cba-423b-8438-2b075017e469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:50.648486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8bf34-4cba-423b-8438-2b075017e469 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:50.661844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 438/868] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:58:50 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-28f8bf34-4cba-423b-8438-2b075017e469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:50.683679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:50.684885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:50.685882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:50.685882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:50.685882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:50.686384 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 431/869] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:50.692344 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:50.840693 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37be6-7f31-4835-a9d7-7d1d02051763 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:50.840895 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37be6-7f31-4835-a9d7-7d1d02051763 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:50.841146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e37be6-7f31-4835-a9d7-7d1d02051763 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:50.842920 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 432/870] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:58:50.848712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f68e-0544-4108-9b32-bfd76e785a0e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:50.849016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f68e-0544-4108-9b32-bfd76e785a0e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:50.849278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345f68e-0544-4108-9b32-bfd76e785a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:50.875393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8ecc-206b-44f3-b6d9-10f537e8a8d9 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:50.879116 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a345f68e-0544-4108-9b32-bfd76e785a0e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:50.881002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a345f68e-0544-4108-9b32-bfd76e785a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:50.889792 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:50.911086 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:50.917767 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:50.918796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:50.919454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:50.919943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:50.920852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:50.921399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:50.923074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:50.939745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 439/871] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:50.948492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:50.985862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 433/872] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:50 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:58:51.123198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee6b7d-9b7d-4ac3-9d14-8166b3616357 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:51.190428 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:51.192005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:51.192786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:51.193439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:51.207900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:51.211110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:51.231451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:51.233139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:51.235325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:51.239305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 440/873] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:51.247718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345f68e-0544-4108-9b32-bfd76e785a0e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:51.260299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:51.342955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:51.412744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082111-fe3f-468d-9b8a-ef76b1c2bb7f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:51.412926 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345f68e-0544-4108-9b32-bfd76e785a0e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:51.473678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:51.474617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:51.474913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:51.475158 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:51.475990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:51.476301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:51.478023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:51.480978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:51.482381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:51.482715 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:51.482972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:51.483582 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:51.484973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:51.493577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:51.497877 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 434/874] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:51.507970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:51.509031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:51.522175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:51.524099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 441/875] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:58:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:51.528862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:51.633880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b30b8-55eb-4cc7-8571-5e4714e0244c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:51.672069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:51.672587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:51.672870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:51.673147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:51.673932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:51.674243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:51.675726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:51.690052 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 435/876] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:51 2022] GET /identity/v3/role_assignments?user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:51.750244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e0206-5616-4584-a357-13dd0f54f58e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:51.781461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:51.824138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d66d57-00d0-4858-884d-71ead360cea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:51.825501 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d66d57-00d0-4858-884d-71ead360cea5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:51.826246 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d66d57-00d0-4858-884d-71ead360cea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:51.826993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d66d57-00d0-4858-884d-71ead360cea5 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:51.828364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d66d57-00d0-4858-884d-71ead360cea5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:51.829141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d66d57-00d0-4858-884d-71ead360cea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:51.832018 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d66d57-00d0-4858-884d-71ead360cea5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:51.846019 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 442/877] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:58:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:51.850945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:51.969513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:52.031475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa8a19-c77b-4076-86fd-8a85216021e2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:52.071531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00e35d-9448-4ccf-99d0-0734508ec719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:52.072154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00e35d-9448-4ccf-99d0-0734508ec719 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:52.072438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00e35d-9448-4ccf-99d0-0734508ec719 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:52.072716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00e35d-9448-4ccf-99d0-0734508ec719 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:52.073366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00e35d-9448-4ccf-99d0-0734508ec719 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:52.073704 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00e35d-9448-4ccf-99d0-0734508ec719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:52.075097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00e35d-9448-4ccf-99d0-0734508ec719 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:52.080662 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 436/878] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:58:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:52.116595 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e454ba-bf05-47c4-af74-1148a39aa646 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:52.117428 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e454ba-bf05-47c4-af74-1148a39aa646 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:52.117571 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e454ba-bf05-47c4-af74-1148a39aa646 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:52.120573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 443/879] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:52.130594 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975930b0-873d-4731-b48c-0d505577147f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:52.131088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975930b0-873d-4731-b48c-0d505577147f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:52.131566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975930b0-873d-4731-b48c-0d505577147f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:52.163650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-975930b0-873d-4731-b48c-0d505577147f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:52.165817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-975930b0-873d-4731-b48c-0d505577147f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:52.266267 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 437/880] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:52.765636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e454ba-bf05-47c4-af74-1148a39aa646 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:52.958183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e454ba-bf05-47c4-af74-1148a39aa646 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:53.018954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b675f-0c7a-425d-8863-4d8edc52dbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:53.019528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b675f-0c7a-425d-8863-4d8edc52dbf4 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:53.019813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b675f-0c7a-425d-8863-4d8edc52dbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:53.020089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b675f-0c7a-425d-8863-4d8edc52dbf4 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:53.020749 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b675f-0c7a-425d-8863-4d8edc52dbf4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:53.022165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b675f-0c7a-425d-8863-4d8edc52dbf4 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:53.099299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 444/881] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:58:52 2022] POST /identity/v3/services => generated 269 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:58:53.835568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b38d-88dd-4637-9f36-4a02e6bf7519 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:53.836395 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b38d-88dd-4637-9f36-4a02e6bf7519 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:53.836543 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b38d-88dd-4637-9f36-4a02e6bf7519 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:53.839624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 438/882] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:53.849288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896329d7-45cd-4082-9aee-d05502b2dc2d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:53.850846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896329d7-45cd-4082-9aee-d05502b2dc2d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:53.851656 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896329d7-45cd-4082-9aee-d05502b2dc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:53.883976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-896329d7-45cd-4082-9aee-d05502b2dc2d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:53.886397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-896329d7-45cd-4082-9aee-d05502b2dc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:53.932969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:54.013862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 445/883] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:54.190001 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbe3f5-d2d8-4d1b-86ea-80b8bff82260 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:54.190635 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbe3f5-d2d8-4d1b-86ea-80b8bff82260 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:54.191129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbe3f5-d2d8-4d1b-86ea-80b8bff82260 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:54.192974 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 439/884] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:54.198980 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1ee6f-a441-4572-a36a-7b3393e2751d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:54.199579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1ee6f-a441-4572-a36a-7b3393e2751d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:54.200085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1ee6f-a441-4572-a36a-7b3393e2751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:54.221031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-f7b1ee6f-a441-4572-a36a-7b3393e2751d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:54.222651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-f7b1ee6f-a441-4572-a36a-7b3393e2751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:54.261073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c113-0477-42ba-b836-a86835e06ce2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:54.261410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c113-0477-42ba-b836-a86835e06ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:54.261642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c113-0477-42ba-b836-a86835e06ce2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:54.263645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 440/885] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:58:54.271144 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cea30-cb75-4dfb-be1e-b168cf2716dd None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:54.271739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cea30-cb75-4dfb-be1e-b168cf2716dd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:54.272294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cea30-cb75-4dfb-be1e-b168cf2716dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:54.295859 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ba5cea30-cb75-4dfb-be1e-b168cf2716dd None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:54.297676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ba5cea30-cb75-4dfb-be1e-b168cf2716dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:54.314128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 446/886] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:54.369756 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:54.385514 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 441/887] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:58:54.529340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1ee6f-a441-4572-a36a-7b3393e2751d None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:54.601985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cea30-cb75-4dfb-be1e-b168cf2716dd None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:54.708021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:54.719563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1ee6f-a441-4572-a36a-7b3393e2751d None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:54.723432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cea30-cb75-4dfb-be1e-b168cf2716dd None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:54.764535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:54.766066 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:54.766782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:54.767485 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:54.768793 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:54.769170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:54.769834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:54.770152 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:54.770462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:54.771141 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:54.772523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:54.773115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:54.790716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:54.792031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:54.793166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:54.797771 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 447/888] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:54.799089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:54.852690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 442/889] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:58:54 2022] POST /identity/v3/services => generated 241 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:58:54.858988 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:54.992412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:55.008618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e545a69-8d35-40e3-b9b1-adffd11f15ab None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:55.103404 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523270-6127-415d-b9c9-2e48c6443bf8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:55.110237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:55.111327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.111827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.112988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.137291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:58:55.150645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:55.151662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:58:55.152504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:55.157638 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 448/890] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Sep 1 11:58:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:55.163507 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:55.178911 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:55.180509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.181320 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.182134 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.184581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:55.185470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:55.189807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:55.199445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:55.205803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 443/891] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:55.215474 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:55.387434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b54e17-813b-4bee-b6ec-3ea53380448a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:55.413887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c064f-b5e0-4cf9-8903-ef6e40b8c3ea None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:55.432213 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:55.458755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167414ae-a22e-41d7-967c-a0312d726db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:55.459389 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167414ae-a22e-41d7-967c-a0312d726db3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.459730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167414ae-a22e-41d7-967c-a0312d726db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.459972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167414ae-a22e-41d7-967c-a0312d726db3 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.460795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167414ae-a22e-41d7-967c-a0312d726db3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:55.461098 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167414ae-a22e-41d7-967c-a0312d726db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:55.462972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167414ae-a22e-41d7-967c-a0312d726db3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:55.482436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 449/892] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Sep 1 11:58:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:55.482436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffacdf-9b21-462b-bd8f-5a710bdaaab4 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.482436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffacdf-9b21-462b-bd8f-5a710bdaaab4 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.482436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffacdf-9b21-462b-bd8f-5a710bdaaab4 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.482436 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 450/893] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:55.484791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8a5f-386c-4bad-a707-6cbf69050b52 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.485305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8a5f-386c-4bad-a707-6cbf69050b52 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.486027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca8a5f-386c-4bad-a707-6cbf69050b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.487259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:55.487888 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.488159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.488429 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.489072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:55.489372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:55.490811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:55.505702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 444/894] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:55.505702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:55.525506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-f2ca8a5f-386c-4bad-a707-6cbf69050b52 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:55.526763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-f2ca8a5f-386c-4bad-a707-6cbf69050b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:55.623531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 451/895] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:58:55.623531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca8a5f-386c-4bad-a707-6cbf69050b52 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:55.636318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:55.656952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194b262-6f4e-4618-9d10-f61f69731642 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:55.702081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:55.702834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.703121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.703392 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.715928 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:55.717488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:55.735203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:55.741428 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:55.742787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:55.742787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:55.742787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:55.742787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 445/896] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:55.743752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:55.779880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca8a5f-386c-4bad-a707-6cbf69050b52 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:55.792186 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:55.830305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:55.831340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.832550 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.833085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.834120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] RBAC: Adding query filter params (interface=public, service_id=6027dea89ad84a5e80534cda2b739a02, region_id=RegionOne) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:55.834657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:55.836312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:55.844432 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 452/897] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 11:58:55 2022] GET /identity/v3/endpoints?service_id=6027dea89ad84a5e80534cda2b739a02&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:55.853933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:55.912208 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2734c-25b6-4fb9-bd7e-b10d9f007216 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:55.952344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:55.953357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:55.954146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:55.954640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:55.955617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:55.956129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:55.957778 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:55.972624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 446/898] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:55.985050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:55.992444 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:56.031650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad677-3fdf-45be-9e09-78242819c55d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:56.081513 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:56.082525 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.083056 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.083510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.084337 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:56.085937 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:56.095055 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 453/899] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:58:55 2022] GET /identity/v3/services => generated 1497 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 11:58:56.098744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:56.154261 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:56.174002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1357-c422-4194-b6a8-35eb5787476f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:56.233965 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:56.235097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.235548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.236098 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.250663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:56.257680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:58:56.282293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:56.284568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:58:56.286812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:56.297692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 447/900] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 11:58:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:56.299779 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:56.358990 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca60b0-7734-4876-9a2b-c6b783db8511 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:56.403617 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:56.406290 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.406978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.407585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.408801 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:56.412659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:56.416419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:56.432528 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:56.433533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:58:56.434620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:56.435530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 454/901] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Sep 1 11:58:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:56.441409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:56.457414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6805ab1-d097-4aec-8f05-0e59cb094fb2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:56.505887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:56.507033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.507882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.508606 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.509579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:56.510251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:56.512319 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:56.539230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 448/902] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Sep 1 11:58:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:56.544603 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:56.548595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:56.584758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2978d73-dafc-4ac6-bad7-75660c8cefd0 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:56.637178 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:56.638070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.638529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.639670 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.639947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:56.640266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:56.641769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:56.648873 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 455/903] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:58:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:56.654303 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:56.704325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d6223-20b6-4cf8-8adf-1656678425cd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:56.750568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:56.750767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9f0ea-057b-4485-b681-3f139379fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:56.751247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9f0ea-057b-4485-b681-3f139379fb58 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.751530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9f0ea-057b-4485-b681-3f139379fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.751803 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9f0ea-057b-4485-b681-3f139379fb58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.825812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5c972-dc93-442b-a9ee-79ad85b3770f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:56.828175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9f0ea-057b-4485-b681-3f139379fb58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:56.829757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9f0ea-057b-4485-b681-3f139379fb58 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:56.831730 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:56.902226 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:56.903823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.904224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.905109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.906607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:56.907471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:56.910334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:56.926605 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 456/904] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:58:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:56.928039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:56.978226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 449/905] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:58:56 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 11:58:56.980455 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40ac8d-532d-4c0d-acf4-415cb8e1dbca None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:56.980949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40ac8d-532d-4c0d-acf4-415cb8e1dbca None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:56.981450 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40ac8d-532d-4c0d-acf4-415cb8e1dbca None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:56.987292 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 450/906] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:56.992220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40ac8d-532d-4c0d-acf4-415cb8e1dbca None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:57.096445 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368d13f-a843-4054-b498-a1472170dec0 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:57.140814 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40ac8d-532d-4c0d-acf4-415cb8e1dbca None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:57.143386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:57.148593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:57.164023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f3d19-cf52-4f7b-a13e-3e05e9edfd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:57.164968 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f3d19-cf52-4f7b-a13e-3e05e9edfd6a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:57.165396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f3d19-cf52-4f7b-a13e-3e05e9edfd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:57.165963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f3d19-cf52-4f7b-a13e-3e05e9edfd6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:57.167443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f3d19-cf52-4f7b-a13e-3e05e9edfd6a None admin] RBAC: Adding query filter params (interface=public, service_id=373bc3887ea94d168294213db78c2e11, region_id=RegionOne) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:57.168007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f3d19-cf52-4f7b-a13e-3e05e9edfd6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:57.171243 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f3d19-cf52-4f7b-a13e-3e05e9edfd6a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:57.180293 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 457/907] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 11:58:56 2022] GET /identity/v3/endpoints?service_id=373bc3887ea94d168294213db78c2e11&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:57.182198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13448-3b08-4ca7-ad83-b8f1e6169449 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:57.182669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13448-3b08-4ca7-ad83-b8f1e6169449 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:57.183118 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec13448-3b08-4ca7-ad83-b8f1e6169449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:57.195527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:57.197325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:57.197325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:57.197735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:57.211322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:57.213045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:57.215553 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-9ec13448-3b08-4ca7-ad83-b8f1e6169449 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:57.218253 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9ec13448-3b08-4ca7-ad83-b8f1e6169449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:58:57.234038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:57.235896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:58:57.238737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:58:57.238737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:58:57.238737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:57.238737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 451/908] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:57.238737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:57.345505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 458/909] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:57.347289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13448-3b08-4ca7-ad83-b8f1e6169449 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:57.386498 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999aeafa-98c7-4833-87bd-827600562050 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:57.427431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:57.428467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:57.428958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:57.429435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:57.430283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:57.432054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:57.439856 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 452/910] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:58:57 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 11:58:57.521581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec13448-3b08-4ca7-ad83-b8f1e6169449 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:57.568561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:57.593357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:57.594435 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:57.594922 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:57.595386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:57.596599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:57.597143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:57.599794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:57.624828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 459/911] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:57.630666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:57.665384 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:57.776594 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c38a0-dffe-4502-95df-1ae2f2774780 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:57.819661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:57.820796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:57.821291 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:57.821874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:57.831422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:57.833417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:58:57.847731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:57.848740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:58:57.849696 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:58:57.850245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:57.852639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 453/912] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:57.859049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:57.954299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:57.979466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c9802-5581-4aeb-b935-34ffbbe32b6b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:58.021030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:58.022213 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:58.022708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:58.023187 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:58.024137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:58.024668 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:58.026418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:58.045929 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3dab6-5f28-44a5-a4f5-2a7b111f6368 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:58.052301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 454/913] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:58.053794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:58.105789 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:58.106523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:58.106835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:58.107107 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:58.108233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:58.109800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:58.126532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:58.127645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:58:58.128526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:58.133741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 460/914] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:58:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:58:58.140523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:58.228995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4798d6-829e-4b0f-ad3c-23e7faae427e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:58.252777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:58.294083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad09d56-c1ec-470f-aaa2-2fdc1f0b65b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:58.295149 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad09d56-c1ec-470f-aaa2-2fdc1f0b65b0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:58.295597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad09d56-c1ec-470f-aaa2-2fdc1f0b65b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:58.296098 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad09d56-c1ec-470f-aaa2-2fdc1f0b65b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:58.297281 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad09d56-c1ec-470f-aaa2-2fdc1f0b65b0 None admin] RBAC: Adding query filter params (role.id=25ca9df14c1447128eda0c9db881eb51, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:58.297821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad09d56-c1ec-470f-aaa2-2fdc1f0b65b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:58.300367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad09d56-c1ec-470f-aaa2-2fdc1f0b65b0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:58.309067 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 455/915] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:58:58 2022] GET /identity/v3/role_assignments?user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=25ca9df14c1447128eda0c9db881eb51 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:58.311661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7d369-ceeb-49cd-ad26-05166e8d88a9 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:58.312243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7d369-ceeb-49cd-ad26-05166e8d88a9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:58.312749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7d369-ceeb-49cd-ad26-05166e8d88a9 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:58.320740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 456/916] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:58:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:58:58.323085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35909-cb6c-4f83-a209-6e747cd9a861 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:58.323643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35909-cb6c-4f83-a209-6e747cd9a861 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:58.324274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35909-cb6c-4f83-a209-6e747cd9a861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:58.350187 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:58.361112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f2aa-bb57-4091-bfcf-0209cbb8a8d8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:58.362656 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2ab35909-cb6c-4f83-a209-6e747cd9a861 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:58:58.367482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2ab35909-cb6c-4f83-a209-6e747cd9a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:58:58.428160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:58.429363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:58.429945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:58.430478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:58.431576 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:58.432154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:58.434607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:58.457320 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 461/917] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:58:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:58.459612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:58.553786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 457/918] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:58:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:58:58.669675 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2ef97-246f-4bc5-877b-442553f38185 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:58.695468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:58.742594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:58.743645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:58.744094 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:58.744589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:58.745760 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:58.746364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:58.749720 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:58.760090 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 462/919] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:58:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:58.771291 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35909-cb6c-4f83-a209-6e747cd9a861 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:58.900098 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:58.972024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35909-cb6c-4f83-a209-6e747cd9a861 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:59.016667 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c70b44-ff8d-4fac-b343-542a00c6b9fb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:59.029853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:59.038900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:59.039781 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:59.044964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:59.044964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:59.044964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] RBAC: Adding query filter params (interface=public, service_id=8ecec3989aca49eea60828eb02c6e8a6, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:59.044964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:59.046094 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:59.059449 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:59.060431 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:59.060890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:59.061290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 458/920] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 11:58:58 2022] GET /identity/v3/endpoints?service_id=8ecec3989aca49eea60828eb02c6e8a6&interface=public®ion_id=RegionOne => generated 194 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:58:59.061823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:59.062984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:59.064640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:59.070496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:58:59.075458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:58:59.076547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 11:58:59.077408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:58:59.086139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 463/921] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:58:59.087868 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:59.134321 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:59.215646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c792c09-ad56-459e-a304-8c2756b13b91 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:59.258089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:59.258777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:59.259064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:59.259344 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:59.260042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:59.260354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:59.261840 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:59.277638 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 464/922] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:59.281499 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:59.337648 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071dbc6-4fd9-4de4-9f52-dfa6946f803d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:59.379783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:59.380415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:59.380760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:59.380972 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:59.381590 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:59.383013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:59.392295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 459/923] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:58:59 2022] GET /identity/v3/services => generated 1497 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 11:58:59.438665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:59.442859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98372-241d-424b-81bf-a8a771eb5967 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:59.484183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae107103-f0e8-4815-be5e-30b6c6408a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:59.484911 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107103-f0e8-4815-be5e-30b6c6408a81 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:59.485192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107103-f0e8-4815-be5e-30b6c6408a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:59.485470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107103-f0e8-4815-be5e-30b6c6408a81 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:59.486171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae107103-f0e8-4815-be5e-30b6c6408a81 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:58:59.486481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae107103-f0e8-4815-be5e-30b6c6408a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:59.487898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae107103-f0e8-4815-be5e-30b6c6408a81 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:59.494210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 465/924] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Sep 1 11:58:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:58:59.504285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:58:59.621075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:58:59.697262 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29752665-e3e3-46ad-8dc5-afbcc8a9ba2f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:58:59.762686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7084b-137c-451c-a0d2-a278cea30759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:58:59.764138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7084b-137c-451c-a0d2-a278cea30759 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:58:59.764918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7084b-137c-451c-a0d2-a278cea30759 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:58:59.765645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7084b-137c-451c-a0d2-a278cea30759 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:58:59.767217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7084b-137c-451c-a0d2-a278cea30759 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:58:59.770113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7084b-137c-451c-a0d2-a278cea30759 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:58:59.871267 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 460/925] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:58:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:00.401242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905815dc-e363-41fb-b351-d01a34b65552 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:00.402087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905815dc-e363-41fb-b351-d01a34b65552 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:00.402087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905815dc-e363-41fb-b351-d01a34b65552 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:00.403687 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 466/926] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:00.411091 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27334-d648-480a-8b0f-132ed8eccca2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:00.412542 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27334-d648-480a-8b0f-132ed8eccca2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:00.413306 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27334-d648-480a-8b0f-132ed8eccca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:00.457096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-73a27334-d648-480a-8b0f-132ed8eccca2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:00.459613 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-73a27334-d648-480a-8b0f-132ed8eccca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:00.470126 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd7df7-4c14-4d33-9a84-043ff6fd9073 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:00.470618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd7df7-4c14-4d33-9a84-043ff6fd9073 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:00.471107 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd7df7-4c14-4d33-9a84-043ff6fd9073 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:00.473972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 467/927] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:00.483332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66378-56f2-4ebc-ae60-1d7bf06098bc None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:00.483822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66378-56f2-4ebc-ae60-1d7bf06098bc None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:00.484316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66378-56f2-4ebc-ae60-1d7bf06098bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:00.514824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-4dc66378-56f2-4ebc-ae60-1d7bf06098bc None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:00.516900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-4dc66378-56f2-4ebc-ae60-1d7bf06098bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:00.610757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 461/928] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:00.651894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 468/929] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:00.883890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a27334-d648-480a-8b0f-132ed8eccca2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:00.930580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:00.993591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a27334-d648-480a-8b0f-132ed8eccca2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.032454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.033115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.033396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.033698 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.045646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:01.050744 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:01.051685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:01.052529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:01.056899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 462/930] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Sep 1 11:59:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:01.058433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66378-56f2-4ebc-ae60-1d7bf06098bc None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.160474 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.199026 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:01.212329 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66378-56f2-4ebc-ae60-1d7bf06098bc None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.276686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.278135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.278796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.278796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.280073 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:01.280626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:01.282249 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:01.293442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 469/931] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:01.300391 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.305876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1d388-aef8-41ff-a716-a4bfc3c3a159 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.309072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:01.376413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.377738 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.378294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.378893 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.380045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:01.382896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:01.400753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:01.401826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 11:59:01.402753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:01.403806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 463/932] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:01.410559 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.443417 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59a5ce-2ecd-4810-a39d-d422ad9a1aed None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.497838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4231ed-98a5-45f3-b1ee-c3f36791bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.499016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4231ed-98a5-45f3-b1ee-c3f36791bf1e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.499295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4231ed-98a5-45f3-b1ee-c3f36791bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.499565 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4231ed-98a5-45f3-b1ee-c3f36791bf1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.515835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4231ed-98a5-45f3-b1ee-c3f36791bf1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:01.517953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4231ed-98a5-45f3-b1ee-c3f36791bf1e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-1e4231ed-98a5-45f3-b1ee-c3f36791bf1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:01.534693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:01.535718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:01.536691 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:01.536691 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:01.536691 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:01.537294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a8729-0610-451e-80a9-8a3c0b43a087 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.543894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 470/933] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:01.543894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439356b-61a8-4654-9cfa-9c2e5a03504a None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.544141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439356b-61a8-4654-9cfa-9c2e5a03504a None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.544422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439356b-61a8-4654-9cfa-9c2e5a03504a None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.546584 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 471/934] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:01.547176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439356b-61a8-4654-9cfa-9c2e5a03504a None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.563084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:01.581192 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa19ca7-fa73-4cc6-94b5-78a852bef165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.583371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa19ca7-fa73-4cc6-94b5-78a852bef165 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.583371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa19ca7-fa73-4cc6-94b5-78a852bef165 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.583371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa19ca7-fa73-4cc6-94b5-78a852bef165 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.583371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa19ca7-fa73-4cc6-94b5-78a852bef165 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:01.583371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa19ca7-fa73-4cc6-94b5-78a852bef165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:01.584789 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa19ca7-fa73-4cc6-94b5-78a852bef165 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:01.590760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 464/935] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:01.592660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e42a6b-33e5-4257-9734-db68c7c05c6f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.592945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e42a6b-33e5-4257-9734-db68c7c05c6f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.593222 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e42a6b-33e5-4257-9734-db68c7c05c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.613251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-70e42a6b-33e5-4257-9734-db68c7c05c6f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:01.614555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-70e42a6b-33e5-4257-9734-db68c7c05c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:01.641287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:01.671931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439356b-61a8-4654-9cfa-9c2e5a03504a None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.692328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 465/936] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:01.692992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e42a6b-33e5-4257-9734-db68c7c05c6f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.715476 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.716096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.716375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.716649 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.717346 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:01.717734 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:01.719371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:01.738039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 472/937] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:01.740038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.826660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:01.840485 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e42a6b-33e5-4257-9734-db68c7c05c6f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.866775 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed166133-0b81-41d9-af62-9539ada932bf None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:01.887358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.887976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.888254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.888526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.889885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:01.890244 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:01.892037 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:01.901772 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 466/938] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:01.905341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:01.918450 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:01.920101 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:01.921253 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:01.921774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:01.932195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:01.934507 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:01.935171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:59:01.952686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:01.953746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:59:01.954716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:01.957526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 473/939] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:01.963637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.028384 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a54e7-53b4-4f69-9c7a-23751d8dade9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.066897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.067525 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.067807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.068082 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.069049 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.070490 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.090835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966b2b-0377-4182-b2bd-bd43a3cbba3b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.119035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:02.127224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.127839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.129998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.129998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.129998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:02.129998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.130837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.150218 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 474/940] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Sep 1 11:59:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:02.151038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.160973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 467/941] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:59:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:59:02.161813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.274023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:02.314970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed8dd4-733e-42bc-a0d0-85c403b455e2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.319399 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6697ae-3261-4e09-80ee-3e45e1f27429 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.369081 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.370262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.371092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.371806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.372692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.374568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.376247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.376910 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.377185 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.377457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:02.386827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:02.387924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:02.389401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:02.396050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 475/942] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:59:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:02.396232 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.397197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.398362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:02.423218 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:02.424258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:02.425190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:02.425190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:02.425190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:02.425190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 468/943] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:02.434731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.466390 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:02.531679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94401cf-2b65-4822-9f7a-9de2af7e85fe None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.569346 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4328-f426-4653-acb4-2cfa0174736e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.577270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.577932 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.578213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.578487 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.579133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:02.579434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.580814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.588184 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 476/944] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:59:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:02.594068 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.608353 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:02.619430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.620410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.621128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.622175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.623078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:02.624178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.626502 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.648778 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 469/945] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:02.650575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.734523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd17a0-bc91-43f5-a9ed-abe2d905a64f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.750835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:02.774424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89fd7f-6f9f-4720-8300-4e9db4e359d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.775335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fd7f-6f9f-4720-8300-4e9db4e359d8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.775771 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fd7f-6f9f-4720-8300-4e9db4e359d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.776198 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89fd7f-6f9f-4720-8300-4e9db4e359d8 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.776681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9950f8b-99ef-47c1-8de5-3d8724cd8a4b None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.777254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89fd7f-6f9f-4720-8300-4e9db4e359d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:02.777871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89fd7f-6f9f-4720-8300-4e9db4e359d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.780014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89fd7f-6f9f-4720-8300-4e9db4e359d8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.786153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 477/946] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:59:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:02.790630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1fa35-f148-4e6c-be4a-6cfd2da1ac2d None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.791153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1fa35-f148-4e6c-be4a-6cfd2da1ac2d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.791438 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1fa35-f148-4e6c-be4a-6cfd2da1ac2d None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.792990 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 478/947] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:02.793685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1fa35-f148-4e6c-be4a-6cfd2da1ac2d None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.827183 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb44e61-f554-47cf-8b04-559f6b779758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.827752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb44e61-f554-47cf-8b04-559f6b779758 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.828042 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb44e61-f554-47cf-8b04-559f6b779758 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.828466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb44e61-f554-47cf-8b04-559f6b779758 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.836336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb44e61-f554-47cf-8b04-559f6b779758 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.839108 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb44e61-f554-47cf-8b04-559f6b779758 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9bb44e61-f554-47cf-8b04-559f6b779758 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:59:02.850902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:02.852741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:02.855087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 470/948] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:02.858726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db13a6-06e6-440a-8fdd-9b76caed05f3 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.859184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db13a6-06e6-440a-8fdd-9b76caed05f3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.859425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db13a6-06e6-440a-8fdd-9b76caed05f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.879984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e9db13a6-06e6-440a-8fdd-9b76caed05f3 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:02.881256 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e9db13a6-06e6-440a-8fdd-9b76caed05f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:02.921090 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1fa35-f148-4e6c-be4a-6cfd2da1ac2d None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:02.938213 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:02.965081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 471/949] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:02.965800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db13a6-06e6-440a-8fdd-9b76caed05f3 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:02.989594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:02.990564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:02.991009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:02.991439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:02.992298 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:02.994961 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:03.087507 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 479/950] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:59:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:03.103684 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db13a6-06e6-440a-8fdd-9b76caed05f3 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:03.183958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:03.184947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:03.185432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:03.186556 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:03.187475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:03.187961 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:03.189679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:03.210133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 472/951] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:03.212608 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:03.256156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:03.337822 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4c0b-4611-4b1d-b314-1c3361527189 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:03.378399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:03.379398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:03.379861 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:03.380351 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/4e0eeaf94d9749c7b0dc15a97a3bf81a/roles/25ca9df14c1447128eda0c9db881eb51` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:03.383071 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=4e0eeaf94d9749c7b0dc15a97a3bf81a, role_id=25ca9df14c1447128eda0c9db881eb51)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:03.384849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:03.484698 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:03.517830 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 480/952] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:59:03 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/4e0eeaf94d9749c7b0dc15a97a3bf81a/roles/25ca9df14c1447128eda0c9db881eb51 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:59:03.560356 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:03.704085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adceb5-d329-451f-98a8-a91a753efbd5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:03.774253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:03.775366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:03.775863 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:03.776334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:03.777465 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:03.780790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:03.785408 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: WARNING keystone.server.flask.common [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 11:59:03.868936 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.sql.core [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 11:59:03.868936 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 11:59:03.868936 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [parameters: {'id': 'e71bf2d4986d471ca64b89ba2e9fb040', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 11:59:03.868936 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81862) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e71bf2d4986d471ca64b89ba2e9fb040.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e71bf2d4986d471ca64b89ba2e9fb040. Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application conn.query(q) Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 11:59:03.897522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application result.read() Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 11:59:03.898889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application next(self.gen) Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application yield resource Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application next(self.gen) Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise self.value Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application session.commit() Sep 01 11:59:03.900437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self.session.flush() Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application compat.raise_( Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 11:59:03.901713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 11:59:03.903039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application conn.query(q) Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application result.read() Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 11:59:03.904333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [parameters: {'id': 'e71bf2d4986d471ca64b89ba2e9fb040', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:03.911507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:03.914173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e71bf2d4986d471ca64b89ba2e9fb040. Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:03.917009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 473/953] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Sep 1 11:59:03 2022] POST /identity/v3/projects => generated 262 bytes in 416 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:03.923326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:04.069454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:04.079407 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64851754-ecc8-4c8b-b5a8-59c0331db513 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:04.116045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3f74e-e351-48f5-a3fa-e811f7eb1101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:04.116689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f74e-e351-48f5-a3fa-e811f7eb1101 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:04.117013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f74e-e351-48f5-a3fa-e811f7eb1101 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:04.117301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3f74e-e351-48f5-a3fa-e811f7eb1101 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:04.125506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3f74e-e351-48f5-a3fa-e811f7eb1101 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:04.127751 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3f74e-e351-48f5-a3fa-e811f7eb1101 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-5bc3f74e-e351-48f5-a3fa-e811f7eb1101 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:59:04.140201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:04.145170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:04.146192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 481/954] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:04.148873 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:04.277948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:04.289728 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b5922-a35f-4a1d-93d3-55f82e8dd81d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:04.337798 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298562ce-7260-46a1-bf3d-697e3e7e6754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:04.338763 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298562ce-7260-46a1-bf3d-697e3e7e6754 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:04.339245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298562ce-7260-46a1-bf3d-697e3e7e6754 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:04.339711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298562ce-7260-46a1-bf3d-697e3e7e6754 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:04.340664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298562ce-7260-46a1-bf3d-697e3e7e6754 None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:04.341197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298562ce-7260-46a1-bf3d-697e3e7e6754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:04.343159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298562ce-7260-46a1-bf3d-697e3e7e6754 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:04.367855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 474/955] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:59:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:04.448028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea98486-1c81-4c45-af32-6690c340b449 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:04.448426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea98486-1c81-4c45-af32-6690c340b449 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:04.448785 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea98486-1c81-4c45-af32-6690c340b449 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:04.450864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 482/956] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:04.458734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b045-c6a1-4548-a2eb-86eee5fe037d None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:04.459243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b045-c6a1-4548-a2eb-86eee5fe037d None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:04.459737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b045-c6a1-4548-a2eb-86eee5fe037d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:04.492004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e639b045-c6a1-4548-a2eb-86eee5fe037d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:04.494859 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e639b045-c6a1-4548-a2eb-86eee5fe037d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:04.631418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 475/957] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:05.099603 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea98486-1c81-4c45-af32-6690c340b449 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:05.291592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea98486-1c81-4c45-af32-6690c340b449 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:05.292180 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c1fb-dec5-4cc6-8064-cf000f3fb5e3 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.292864 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c1fb-dec5-4cc6-8064-cf000f3fb5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.292864 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c1fb-dec5-4cc6-8064-cf000f3fb5e3 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.294416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 476/958] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:05.301633 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1492c-292a-4e7a-b646-a44df4a4d7bf None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.301882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1492c-292a-4e7a-b646-a44df4a4d7bf None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.302195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1492c-292a-4e7a-b646-a44df4a4d7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.329751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-05e1492c-292a-4e7a-b646-a44df4a4d7bf None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:05.332555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-05e1492c-292a-4e7a-b646-a44df4a4d7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:05.350565 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:05.351108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.351384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.351673 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.352274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:05.354392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:05.365878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:05.367958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:05.369762 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 483/959] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:05.381973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:05.407294 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:05.494821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 477/960] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:05.522766 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32054eaa-a26b-403d-b4b4-e99ffa8ac73c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:05.563259 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:05.563883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.564169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.564455 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.565083 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:05.565389 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:05.566807 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:05.572971 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 484/961] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:59:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:05.582364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1492c-292a-4e7a-b646-a44df4a4d7bf None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:05.604073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:05.747347 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1492c-292a-4e7a-b646-a44df4a4d7bf None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:05.766407 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:05.795600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818fb6-92de-40cf-b739-c0ee0681871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:05.796243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818fb6-92de-40cf-b739-c0ee0681871a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.797195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818fb6-92de-40cf-b739-c0ee0681871a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.797475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818fb6-92de-40cf-b739-c0ee0681871a None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.798446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818fb6-92de-40cf-b739-c0ee0681871a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:05.798767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818fb6-92de-40cf-b739-c0ee0681871a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:05.801092 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818fb6-92de-40cf-b739-c0ee0681871a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:05.808637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 478/962] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:59:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:05.875056 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:05.896462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbe3bf-4568-4795-a2a1-3a60c286308c None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.896726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbe3bf-4568-4795-a2a1-3a60c286308c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.896912 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b89c-8479-4419-8e24-1f3b181fcae1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:05.897081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbe3bf-4568-4795-a2a1-3a60c286308c None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.898938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 479/963] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:05.905734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484fd6c-9de1-4021-96d9-0802b4c5ba78 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.906648 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484fd6c-9de1-4021-96d9-0802b4c5ba78 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.907435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484fd6c-9de1-4021-96d9-0802b4c5ba78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.934806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9484fd6c-9de1-4021-96d9-0802b4c5ba78 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:05.936896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9484fd6c-9de1-4021-96d9-0802b4c5ba78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:05.957075 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:05.958430 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:05.958712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:05.959385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:05.960014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:05.962191 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:05.975396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:05.976526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:05.977386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:05.981751 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 485/964] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:59:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:05.982859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:06.030878 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 480/965] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:06.129841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c60d-efa1-4abc-873b-0ff3b23c7e30 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:06.201859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:06.202942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.203449 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.203941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.205152 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:06.205736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:06.208434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:06.218759 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 486/966] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:59:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:06.224317 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484fd6c-9de1-4021-96d9-0802b4c5ba78 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:06.229195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:06.350894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484fd6c-9de1-4021-96d9-0802b4c5ba78 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:06.377389 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:06.395689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:06.401933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:06.403348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.403832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.404867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.406331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:06.407369 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:06.410101 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:06.425667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 481/967] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:59:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:06.430578 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:06.509570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:06.548294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f7ef5-9e80-48ba-ba6d-ab122af00e53 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:06.553258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b509f-11f4-4bd9-82d9-687a9fb8a8d3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:06.596342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:06.596986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.597263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.597527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.598311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] RBAC: Adding query filter params (interface=admin, service_id=8ecec3989aca49eea60828eb02c6e8a6, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:06.598616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:06.600068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:06.611087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 482/968] 10.222.0.48 () {56 vars in 1249 bytes} [Thu Sep 1 11:59:06 2022] GET /identity/v3/endpoints?service_id=8ecec3989aca49eea60828eb02c6e8a6&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:06.612273 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:06.618410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:06.620114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.620917 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.621816 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.644745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:06.656881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:06.659005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:06.660646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:06.672650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 487/969] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Sep 1 11:59:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:06.676501 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:06.760828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16414fff-ca23-470a-b021-b61fa059d8d8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:06.788736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:06.817092 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:06.826459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94daaea-89fc-40a2-993f-ab4f785ea424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:06.827759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94daaea-89fc-40a2-993f-ab4f785ea424 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.827932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94daaea-89fc-40a2-993f-ab4f785ea424 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.828295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94daaea-89fc-40a2-993f-ab4f785ea424 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.829320 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94daaea-89fc-40a2-993f-ab4f785ea424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:06.831012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d356a5-1e85-44b6-8674-f47f64269add None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:06.831840 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94daaea-89fc-40a2-993f-ab4f785ea424 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:06.844302 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 483/970] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:59:06 2022] GET /identity/v3/services => generated 1497 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 11:59:06.848835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8934942-333c-49ef-a1d4-062c13dcee08 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.850196 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8934942-333c-49ef-a1d4-062c13dcee08 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.850535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8934942-333c-49ef-a1d4-062c13dcee08 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.856543 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 484/971] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:06.858667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086603e1-9621-435c-ba64-372645d38e79 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.858840 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086603e1-9621-435c-ba64-372645d38e79 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.859097 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086603e1-9621-435c-ba64-372645d38e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.886433 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-086603e1-9621-435c-ba64-372645d38e79 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:06.887929 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:06.889140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:06.889428 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:06.889747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:06.890496 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:06.890860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:06.891605 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-086603e1-9621-435c-ba64-372645d38e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:06.892346 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:06.909701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 488/972] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Sep 1 11:59:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:06.914646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:07.008397 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 485/973] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:07.116007 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e856d05-1058-4661-858f-7a4b78c13052 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:07.176289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:07.184322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:07.185340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:07.186469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:07.186909 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:07.207654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:07.210269 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:07.241648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:07.242685 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:07.243594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:07.243594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:07.243594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:07.244188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 489/974] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:07.250617 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086603e1-9621-435c-ba64-372645d38e79 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:07.339987 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:07.364755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086603e1-9621-435c-ba64-372645d38e79 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:07.398214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:07.415473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:07.416960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:07.417780 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:07.419436 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:07.422275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:07.423604 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:07.427915 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:07.459087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 486/975] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:07.468896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:07.499703 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87bceb-9742-456e-98ce-9c4506f5b1f8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:07.541523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:07.542563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:07.543104 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:07.543582 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:07.544450 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:07.546705 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:07.550134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WARNING keystone.server.flask.common [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 11:59:07.594718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.sql.core [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 11:59:07.594718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 11:59:07.594718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [parameters: {'user_id': 'e50c797cc9b54da3a7d5efa63436fa00', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 11:59:07.594718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81861) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application conn.query(q) Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result.read() Sep 01 11:59:07.605736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 11:59:07.606732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application next(self.gen) Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application yield resource Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application next(self.gen) Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise self.value Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application session.commit() Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 11:59:07.607758 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.session.flush() Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application compat.raise_( Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 11:59:07.608660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 11:59:07.610296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application conn.query(q) Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application result.read() Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 11:59:07.611173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e50c797cc9b54da3a7d5efa63436fa00', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:07.612110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 11:59:07.614265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 11:59:07.616468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 11:59:07.616468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 11:59:07.616468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 11:59:07.616468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 11:59:07.616468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 11:59:07.616468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:07.620687 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 490/976] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Sep 1 11:59:07 2022] POST /identity/v3/users => generated 160 bytes in 282 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:07.629506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:07.662891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6109734-fcbb-41da-9203-4cd2a9aa8537 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:07.692714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:07.724174 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:07.724821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:07.725095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:07.725420 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:07.739205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:07.743250 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:59:07.758240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:07.760162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 11:59:07.762074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:07.764645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 487/977] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 11:59:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:07.765671 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:07.830926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ad10c-df89-4348-b13b-b4ecb0e47756 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:07.836287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:07.897252 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af77a6d-d846-458d-ab8d-169889019834 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:07.899043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:07.901865 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:07.902714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:07.903428 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:07.921707 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:07.925287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:07.941303 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:07.942129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:07.942508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:07.942960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:07.943990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:07.944462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:07.946156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:59:07.953410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:07.955607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 11:59:07.958215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:07.970139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 491/978] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:59:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:07.971449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 488/979] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Sep 1 11:59:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:07.973045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:07.977187 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:08.096392 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cb125-96d5-4378-b8ad-26b9fc28b928 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:08.100292 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:08.133914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:08.134529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.134804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.135073 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.148367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f54e1b-9556-4d4e-aafd-60f8eb0442e3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:08.162613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:08.196710 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ffdc9-9393-4ba5-972c-b69296701ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:08.198181 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffdc9-9393-4ba5-972c-b69296701ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.198499 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffdc9-9393-4ba5-972c-b69296701ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.198761 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffdc9-9393-4ba5-972c-b69296701ec3 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.199432 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ffdc9-9393-4ba5-972c-b69296701ec3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:08.199738 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ffdc9-9393-4ba5-972c-b69296701ec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:08.201267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ffdc9-9393-4ba5-972c-b69296701ec3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:08.215578 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:08.218047 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:08.242607 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 492/980] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Sep 1 11:59:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:08.244224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a735236-6eed-4ce8-8a03-2ce3b6d2eca3 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.244688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a735236-6eed-4ce8-8a03-2ce3b6d2eca3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.244963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a735236-6eed-4ce8-8a03-2ce3b6d2eca3 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.246524 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 493/981] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:08.252683 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ee9e3-7655-4e0d-b095-341b97952267 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.253078 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ee9e3-7655-4e0d-b095-341b97952267 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.253304 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ee9e3-7655-4e0d-b095-341b97952267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.274017 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-ef9ee9e3-7655-4e0d-b095-341b97952267 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:08.275614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-ef9ee9e3-7655-4e0d-b095-341b97952267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:08.390045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 494/982] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:08.405074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 489/983] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 11:59:08.414811 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ee9e3-7655-4e0d-b095-341b97952267 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:08.580747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:08.591960 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ee9e3-7655-4e0d-b095-341b97952267 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:08.652737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:08.654876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.655627 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.656353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.668508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:08.674800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:08.676280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:08.690072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:08.698280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:08.707467 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:08.709138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:08.709138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:08.709138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:08.709138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 495/984] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:08.709138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:08.834012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36e1e-805c-42d8-bb94-cec15de0103f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:08.843249 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4531d6-ca94-4b98-ad84-854c4457c7a9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:08.889699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:08.890702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a9e9b-a859-47fa-8696-70960530d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:08.890959 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.891452 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.891725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a9e9b-a859-47fa-8696-70960530d487 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.892137 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.892254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a9e9b-a859-47fa-8696-70960530d487 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.892691 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a9e9b-a859-47fa-8696-70960530d487 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.893333 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:08.893899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a9e9b-a859-47fa-8696-70960530d487 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:08.894130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:08.895786 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:08.896313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a9e9b-a859-47fa-8696-70960530d487 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:08.910702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 496/985] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:08.917242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:08.932258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:08.978388 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 490/986] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:59:08 2022] POST /identity/v3/services => generated 242 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:08.979834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac84923-e297-482b-b056-f43fc0710c5f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.980223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac84923-e297-482b-b056-f43fc0710c5f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.980622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac84923-e297-482b-b056-f43fc0710c5f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:08.982239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 491/987] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:08.987900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6b9a0-9f3e-4e5d-aa7c-80fed77237e2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:08.988183 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6b9a0-9f3e-4e5d-aa7c-80fed77237e2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:08.988462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6b9a0-9f3e-4e5d-aa7c-80fed77237e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:09.009664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-80d6b9a0-9f3e-4e5d-aa7c-80fed77237e2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:09.011540 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-80d6b9a0-9f3e-4e5d-aa7c-80fed77237e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:09.069230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0191f-f80b-40fa-8df0-46e70aae0e4a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:09.098173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:09.121551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 492/988] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:09.126092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:09.127244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:09.127753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:09.128196 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:09.138407 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:09.140660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:59:09.155874 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:09.157921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:09.159790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 497/989] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:09.164012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6b9a0-9f3e-4e5d-aa7c-80fed77237e2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:09.295385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6b9a0-9f3e-4e5d-aa7c-80fed77237e2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:09.348035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:09.349241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:09.349886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:09.350457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:09.351790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:09.352448 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:09.355176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:09.382431 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 493/990] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:09.383945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:09.399509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:09.462251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:09.475038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:09.527437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964781-0fc1-48ca-b5d9-4d5f87f20e93 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:09.537809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f08db6-9d19-4576-88e4-4505d77ee14a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:09.568121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:09.569133 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:09.569489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:09.569853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:09.570519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:09.572117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:09.582390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:09.583452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:09.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 494/991] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Sep 1 11:59:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:09.593429 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:09.598569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:09.599190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:09.599469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:09.599742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:09.600483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] RBAC: Adding query filter params (role.id=25ca9df14c1447128eda0c9db881eb51, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:09.600795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:09.602345 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:09.608880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 498/992] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:09 2022] GET /identity/v3/role_assignments?user.id=4e0eeaf94d9749c7b0dc15a97a3bf81a&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=25ca9df14c1447128eda0c9db881eb51 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:09.718325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499acf4-1f74-4a5f-9f88-b55ca67b1c7d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:09.734322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:09.756627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:09.757258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:09.757537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:09.757834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:09.758495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:09.758799 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:09.760207 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:09.772899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 495/993] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:59:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:09.773127 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:09.907551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd271b-ab07-4191-8d01-e2b67b8d598d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:09.937142 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:09.946732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249fb05-e7b5-4260-8181-7d5864966de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:09.947352 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249fb05-e7b5-4260-8181-7d5864966de3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:09.951666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249fb05-e7b5-4260-8181-7d5864966de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:09.951666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249fb05-e7b5-4260-8181-7d5864966de3 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:09.951666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249fb05-e7b5-4260-8181-7d5864966de3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:09.951666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249fb05-e7b5-4260-8181-7d5864966de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:09.951666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249fb05-e7b5-4260-8181-7d5864966de3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:09.962464 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 499/994] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Sep 1 11:59:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:09.966396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:10.155356 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e055d-a9f4-4b2f-a9a7-20dda262b405 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:10.221432 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71abc-0011-4ad5-b39b-7e12abbc15d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:10.222160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71abc-0011-4ad5-b39b-7e12abbc15d2 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:10.222654 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71abc-0011-4ad5-b39b-7e12abbc15d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:10.223104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71abc-0011-4ad5-b39b-7e12abbc15d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:10.224380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71abc-0011-4ad5-b39b-7e12abbc15d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:10.227179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71abc-0011-4ad5-b39b-7e12abbc15d2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:10.321661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 496/995] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:59:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:11.020993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dca0-f02d-4567-988c-6fd406e2c9f2 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:11.021902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dca0-f02d-4567-988c-6fd406e2c9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:11.021902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dca0-f02d-4567-988c-6fd406e2c9f2 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:11.023220 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 500/996] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:11.028730 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d9a5-bfb7-4580-8851-a803a7018a05 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:11.030115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d9a5-bfb7-4580-8851-a803a7018a05 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:11.030115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d9a5-bfb7-4580-8851-a803a7018a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:11.052277 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-9297d9a5-bfb7-4580-8851-a803a7018a05 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:11.053811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9297d9a5-bfb7-4580-8851-a803a7018a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:11.089165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:11.146134 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 497/997] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:11.414348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513dca0-f02d-4567-988c-6fd406e2c9f2 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:11.595064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513dca0-f02d-4567-988c-6fd406e2c9f2 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:11.635583 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f91dd-72da-4f88-94e3-e289fb8ff8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:11.636631 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f91dd-72da-4f88-94e3-e289fb8ff8ca None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:11.637138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f91dd-72da-4f88-94e3-e289fb8ff8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:11.637719 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f91dd-72da-4f88-94e3-e289fb8ff8ca None admin] PATH_INFO: `/v3/services/compute` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:11.638564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f91dd-72da-4f88-94e3-e289fb8ff8ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:11.640158 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f91dd-72da-4f88-94e3-e289fb8ff8ca None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-ca9f91dd-72da-4f88-94e3-e289fb8ff8ca None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:11.649732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:11.650856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:11.651782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:11.652554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 501/998] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:11.659010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d9a5-bfb7-4580-8851-a803a7018a05 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:11.730755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe40d-2707-45b0-bb03-ed24f9c38567 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:11.731090 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe40d-2707-45b0-bb03-ed24f9c38567 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:11.731336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbe40d-2707-45b0-bb03-ed24f9c38567 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:11.733330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 502/999] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:11.738715 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb87ab-01ac-44bb-ac2b-4e48084d0f68 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:11.739011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb87ab-01ac-44bb-ac2b-4e48084d0f68 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:11.744318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb87ab-01ac-44bb-ac2b-4e48084d0f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:11.758402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-aaeb87ab-01ac-44bb-ac2b-4e48084d0f68 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:11.759628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-aaeb87ab-01ac-44bb-ac2b-4e48084d0f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:11.804933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d9a5-bfb7-4580-8851-a803a7018a05 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:11.820794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:11.846423 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 503/1000] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:11.858576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:11.859203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:11.859507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:11.859759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:11.860452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:11.860783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:11.862391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:11.876410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 498/1001] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:59:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:11.877993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb87ab-01ac-44bb-ac2b-4e48084d0f68 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:12.004001 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb87ab-01ac-44bb-ac2b-4e48084d0f68 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:12.056535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:12.057481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.057939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.058364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.059403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:12.059852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:12.062254 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:12.076595 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 504/1002] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:59:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:12.078240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:12.120292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:12.124482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:12.190409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:12.213110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a097ed3-ccfd-4541-becb-3a2a9264fb51 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:12.257032 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:12.257758 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.258114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.258552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.260964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] RBAC: Adding query filter params (interface=public, service_id=91c15e3d76e341bd8ac37ed309e45841, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:12.260964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:12.263537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:12.270342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 499/1003] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 11:59:12 2022] GET /identity/v3/endpoints?service_id=91c15e3d76e341bd8ac37ed309e45841&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:12.272655 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b694eca-56f8-461e-9d81-80bcf633ed6e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:12.276425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:12.320136 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:12.321150 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.321642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.322016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] PATH_INFO: `/v3/services/image` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.322753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:12.325179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:12.340833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:12.343419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:12.344728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 505/1004] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Sep 1 11:59:12 2022] GET /identity/v3/services/image => generated 86 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:12.349588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:12.433165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:12.448975 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b7aba-4e6e-4966-9c4e-ee39b4e84fcc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:12.494483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:12.495706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.496361 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.496880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.498514 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:12.500625 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bceda-0a4e-4735-8b5d-edce9bcc6448 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:12.500876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:12.521735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 500/1005] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:59:12 2022] GET /identity/v3/services => generated 1727 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 11:59:12.531522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:12.558851 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76bd3b-ae86-4564-9eb2-b83dd6e0fb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:12.559481 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76bd3b-ae86-4564-9eb2-b83dd6e0fb40 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.559756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76bd3b-ae86-4564-9eb2-b83dd6e0fb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.560027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76bd3b-ae86-4564-9eb2-b83dd6e0fb40 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.560735 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76bd3b-ae86-4564-9eb2-b83dd6e0fb40 None admin] RBAC: Adding query filter params (name=image) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:12.561010 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76bd3b-ae86-4564-9eb2-b83dd6e0fb40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:12.562492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76bd3b-ae86-4564-9eb2-b83dd6e0fb40 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:12.569914 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 506/1006] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:12.575524 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:12.705273 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3061f5e-49a8-4b1c-8f39-4b3212f25b70 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:12.750185 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80975d-5941-4030-aeec-bce852ecddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:12.750798 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80975d-5941-4030-aeec-bce852ecddff None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.751071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80975d-5941-4030-aeec-bce852ecddff None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.751338 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80975d-5941-4030-aeec-bce852ecddff None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.751969 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80975d-5941-4030-aeec-bce852ecddff None admin] RBAC: Adding query filter params (type=image) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:12.752270 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80975d-5941-4030-aeec-bce852ecddff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:12.753669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80975d-5941-4030-aeec-bce852ecddff None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:12.759869 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 501/1007] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:12.796335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85fc0c-39a7-4f82-8e58-e45e4b9ea979 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.796616 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85fc0c-39a7-4f82-8e58-e45e4b9ea979 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.796755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85fc0c-39a7-4f82-8e58-e45e4b9ea979 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.798598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 507/1008] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:12.804513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649a672-585d-4611-b17f-262282665c2a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:12.804886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649a672-585d-4611-b17f-262282665c2a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:12.805076 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649a672-585d-4611-b17f-262282665c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:12.828833 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-1649a672-585d-4611-b17f-262282665c2a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:12.830142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1649a672-585d-4611-b17f-262282665c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:12.851329 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:12.929739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 502/1009] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:13.207203 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85fc0c-39a7-4f82-8e58-e45e4b9ea979 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:13.383547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85fc0c-39a7-4f82-8e58-e45e4b9ea979 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:13.436033 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317d573-411f-49f0-bbdf-c0c61b983dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:13.437312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317d573-411f-49f0-bbdf-c0c61b983dcd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:13.437864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317d573-411f-49f0-bbdf-c0c61b983dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:13.438439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317d573-411f-49f0-bbdf-c0c61b983dcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:13.440115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317d573-411f-49f0-bbdf-c0c61b983dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:13.442264 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317d573-411f-49f0-bbdf-c0c61b983dcd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:13.447225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 508/1010] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:13.456636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649a672-585d-4611-b17f-262282665c2a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:13.601753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649a672-585d-4611-b17f-262282665c2a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:13.668041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ddf77-e4a1-438a-bd31-1425ab577558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:13.669191 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ddf77-e4a1-438a-bd31-1425ab577558 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:13.669585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ddf77-e4a1-438a-bd31-1425ab577558 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:13.670139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ddf77-e4a1-438a-bd31-1425ab577558 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:13.671685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ddf77-e4a1-438a-bd31-1425ab577558 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:13.674860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ddf77-e4a1-438a-bd31-1425ab577558 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:13.785644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 503/1011] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:59:13 2022] POST /identity/v3/projects => generated 311 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:14.562116 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e7f8-ff56-4716-b090-09da5fd9962d None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:14.562695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e7f8-ff56-4716-b090-09da5fd9962d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:14.563075 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74e7f8-ff56-4716-b090-09da5fd9962d None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:14.565599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 509/1012] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:14.571889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00e4bf-3ee5-4e2f-9f03-f862592f2f33 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:14.572680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00e4bf-3ee5-4e2f-9f03-f862592f2f33 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:14.572680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00e4bf-3ee5-4e2f-9f03-f862592f2f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:14.579805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:14.598946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ba00e4bf-3ee5-4e2f-9f03-f862592f2f33 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:14.600964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ba00e4bf-3ee5-4e2f-9f03-f862592f2f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:14.723962 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 504/1013] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:14.824491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a750-9310-4ec6-8227-e7e0feb23a4d None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:14.824859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a750-9310-4ec6-8227-e7e0feb23a4d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:14.825042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18a750-9310-4ec6-8227-e7e0feb23a4d None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:14.827274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 510/1014] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:14.832853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbdf3c-2bc6-4c58-84e4-45d42c805e07 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:14.833250 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbdf3c-2bc6-4c58-84e4-45d42c805e07 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:14.833437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbdf3c-2bc6-4c58-84e4-45d42c805e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:14.853065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-91cbdf3c-2bc6-4c58-84e4-45d42c805e07 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:14.854321 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-91cbdf3c-2bc6-4c58-84e4-45d42c805e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:14.932368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 505/1015] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:15.208110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18a750-9310-4ec6-8227-e7e0feb23a4d None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:15.227165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbdf3c-2bc6-4c58-84e4-45d42c805e07 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:15.261054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:15.311632 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:15.359147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18a750-9310-4ec6-8227-e7e0feb23a4d None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:15.370248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbdf3c-2bc6-4c58-84e4-45d42c805e07 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:15.413286 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:15.413646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c6510-2809-4920-a870-576463aad244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:15.414347 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:15.414801 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c6510-2809-4920-a870-576463aad244 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:15.414906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:15.415192 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:15.415305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c6510-2809-4920-a870-576463aad244 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:15.415740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c6510-2809-4920-a870-576463aad244 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:15.416134 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:15.416993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c6510-2809-4920-a870-576463aad244 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:15.418438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:15.419569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c6510-2809-4920-a870-576463aad244 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:15.435041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:15.441343 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:15.442345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 506/1016] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:15.444232 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:15.487366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 511/1017] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:59:15 2022] POST /identity/v3/services => generated 242 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:15.633066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd2993-4fd5-4ce5-9fd8-a136b3ceeb14 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:15.682228 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:15.703719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243c4e5-d3c9-40da-baa3-02918cbc7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:15.704331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243c4e5-d3c9-40da-baa3-02918cbc7d7f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:15.704615 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243c4e5-d3c9-40da-baa3-02918cbc7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:15.704912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243c4e5-d3c9-40da-baa3-02918cbc7d7f None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:15.705537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243c4e5-d3c9-40da-baa3-02918cbc7d7f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:15.706044 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243c4e5-d3c9-40da-baa3-02918cbc7d7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:15.708189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243c4e5-d3c9-40da-baa3-02918cbc7d7f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:15.717165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 507/1018] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:59:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:15.723261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c6510-2809-4920-a870-576463aad244 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:15.886072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c6510-2809-4920-a870-576463aad244 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:15.891179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:15.896571 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926008a2-8cd2-44a9-b2b3-3c13ddab1f19 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:15.896854 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926008a2-8cd2-44a9-b2b3-3c13ddab1f19 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:15.897125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926008a2-8cd2-44a9-b2b3-3c13ddab1f19 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:15.898521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 508/1019] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:15.906768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db21d91-759e-4e71-9536-0a8d4ec3b5d1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:15.907156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db21d91-759e-4e71-9536-0a8d4ec3b5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:15.907433 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db21d91-759e-4e71-9536-0a8d4ec3b5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:15.929395 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-7db21d91-759e-4e71-9536-0a8d4ec3b5d1 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:15.931489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7db21d91-759e-4e71-9536-0a8d4ec3b5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:15.937954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:15.938587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:15.938870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:15.939147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:15.939788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:15.940091 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:15.941516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:15.950078 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 512/1020] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Sep 1 11:59:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:15.953643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:16.028616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 509/1021] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:16.085830 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e30590-efd2-45c4-920d-9a87d06af0fd None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:16.125506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:16.126173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:16.126456 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:16.126738 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:16.127459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:16.128924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:16.238130 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 513/1022] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:59:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:59:16.325906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db21d91-759e-4e71-9536-0a8d4ec3b5d1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:16.478332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db21d91-759e-4e71-9536-0a8d4ec3b5d1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:16.508621 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:16.551223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81f7e3-78bd-41d5-b411-6be0f44abb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:16.552050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f7e3-78bd-41d5-b411-6be0f44abb3d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:16.552464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f7e3-78bd-41d5-b411-6be0f44abb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:16.552894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f7e3-78bd-41d5-b411-6be0f44abb3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:16.554269 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81f7e3-78bd-41d5-b411-6be0f44abb3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:16.555855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81f7e3-78bd-41d5-b411-6be0f44abb3d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:16.558845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 510/1023] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:16.564300 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:16.593428 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:16.737658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763facd-ffe7-4a11-8e7d-53b569e084a2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:16.784504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8b19d-50c3-4351-9c11-7df3b974e32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:16.785120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8b19d-50c3-4351-9c11-7df3b974e32f None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:16.785379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8b19d-50c3-4351-9c11-7df3b974e32f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:16.785809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8b19d-50c3-4351-9c11-7df3b974e32f None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:16.786523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8b19d-50c3-4351-9c11-7df3b974e32f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:16.788787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8b19d-50c3-4351-9c11-7df3b974e32f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:16.876367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 514/1024] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:59:16 2022] POST /identity/v3/users => generated 285 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:59:17.581020 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4003-6807-40c5-89ac-f41802bb3516 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:17.581550 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4003-6807-40c5-89ac-f41802bb3516 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:17.581550 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4003-6807-40c5-89ac-f41802bb3516 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:17.582944 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 511/1025] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:17.588611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8745d8-b311-455e-b488-622bb6fd2848 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:17.588988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8745d8-b311-455e-b488-622bb6fd2848 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:17.589179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8745d8-b311-455e-b488-622bb6fd2848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:17.609044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-7a8745d8-b311-455e-b488-622bb6fd2848 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:17.610280 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7a8745d8-b311-455e-b488-622bb6fd2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:17.684734 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 515/1026] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:17.958504 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4003-6807-40c5-89ac-f41802bb3516 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:18.118836 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4003-6807-40c5-89ac-f41802bb3516 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:18.158584 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085cc04-80d1-4a3c-8727-84acd97e757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:18.159492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085cc04-80d1-4a3c-8727-84acd97e757b None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.159841 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085cc04-80d1-4a3c-8727-84acd97e757b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.160147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085cc04-80d1-4a3c-8727-84acd97e757b None admin] PATH_INFO: `/v3/services/image` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.160846 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085cc04-80d1-4a3c-8727-84acd97e757b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:18.162634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085cc04-80d1-4a3c-8727-84acd97e757b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2085cc04-80d1-4a3c-8727-84acd97e757b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:18.173259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:18.177947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:18.178835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 512/1027] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Sep 1 11:59:17 2022] GET /identity/v3/services/image => generated 86 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:18.181092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8745d8-b311-455e-b488-622bb6fd2848 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:18.199158 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:18.338782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8745d8-b311-455e-b488-622bb6fd2848 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:18.342537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a5894-731f-49b5-b5ff-31f89bf93643 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.342816 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a5894-731f-49b5-b5ff-31f89bf93643 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.343090 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a5894-731f-49b5-b5ff-31f89bf93643 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.344469 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 513/1028] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:18.350031 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe529-27da-47f0-bd75-ab39fee6c717 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.350362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe529-27da-47f0-bd75-ab39fee6c717 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.350593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe529-27da-47f0-bd75-ab39fee6c717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.369826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ca0fe529-27da-47f0-bd75-ab39fee6c717 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:18.370967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ca0fe529-27da-47f0-bd75-ab39fee6c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:18.382050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:18.382886 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.383039 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.383295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.383944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] RBAC: Adding query filter params (name=image) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:18.384455 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:18.385801 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:18.392690 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 516/1029] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:18.398907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:18.435967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:18.478328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 514/1030] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:18.504514 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:18.539796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88190e8c-bc05-4d73-867b-3f8bb95c88df None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:18.586005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:18.586618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.586891 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.587166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.587813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] RBAC: Adding query filter params (type=image) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:18.588118 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:18.589552 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:18.601915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 517/1031] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:18.606323 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe529-27da-47f0-bd75-ab39fee6c717 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:18.741973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe529-27da-47f0-bd75-ab39fee6c717 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:18.755268 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:18.782311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:18.782948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.783837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.783837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.784246 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] RBAC: Adding query filter params (interface=public, service_id=1247e932021040f184619ef46c570858, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:18.784515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:18.785938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:18.791960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 515/1032] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 11:59:18 2022] GET /identity/v3/endpoints?service_id=1247e932021040f184619ef46c570858&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:18.798572 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:18.854738 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:18.887214 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6095ce3-e96d-4325-9bac-a763a648144c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:18.933826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2ef9f-257d-41dc-9a58-daa29290c04f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:18.943053 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:18.944495 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.944713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.945114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.957185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:18.961591 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:18.967368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:18.972380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:18.973533 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 518/1033] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Sep 1 11:59:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:18.976064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:18.993938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:18.995255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:18.995889 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:18.996474 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:18.997274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:18.999136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:19.007961 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 516/1034] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 11:59:18 2022] GET /identity/v3/services => generated 1957 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 11:59:19.138133 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff613-3dcd-41d8-9325-1fb20f86ffdd None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:19.188242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e93cd-e00c-44e0-85ce-1a8d5b711439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:19.189215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e93cd-e00c-44e0-85ce-1a8d5b711439 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:19.189735 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e93cd-e00c-44e0-85ce-1a8d5b711439 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:19.190182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e93cd-e00c-44e0-85ce-1a8d5b711439 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:19.191288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e93cd-e00c-44e0-85ce-1a8d5b711439 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:19.191813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e93cd-e00c-44e0-85ce-1a8d5b711439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:19.194540 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e93cd-e00c-44e0-85ce-1a8d5b711439 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:19.207632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 519/1035] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Sep 1 11:59:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:19.216081 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:19.279887 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a1ac-09d5-4b7f-9ba5-c780584889a7 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:19.280277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a1ac-09d5-4b7f-9ba5-c780584889a7 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:19.280470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6a1ac-09d5-4b7f-9ba5-c780584889a7 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:19.282014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 520/1036] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:19.287120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954852b0-3ce9-4a60-8448-13fb9f6e45df None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:19.287453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954852b0-3ce9-4a60-8448-13fb9f6e45df None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:19.287674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954852b0-3ce9-4a60-8448-13fb9f6e45df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:19.308524 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-954852b0-3ce9-4a60-8448-13fb9f6e45df None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:19.309989 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-954852b0-3ce9-4a60-8448-13fb9f6e45df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:19.378125 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552c776-1d89-4acb-a227-7549393cd8ba None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:19.381401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:19.401713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:19.427745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:19.428723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:19.429205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:19.429720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:19.429953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 521/1037] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:19.448700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:19.451415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:19.471813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:19.479619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:19.481580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:19.481580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:19.481580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:19.481580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 517/1038] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:19.487109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954852b0-3ce9-4a60-8448-13fb9f6e45df None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:19.639863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954852b0-3ce9-4a60-8448-13fb9f6e45df None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:19.690420 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:19.693470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:19.694707 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:19.695176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:19.695636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:19.696712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:19.697222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:19.699734 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:19.722623 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 522/1039] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:19.727383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:19.784748 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:19.842465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427fd89-aba6-42ca-92b2-22dfdddf8318 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:19.870427 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731e09e-4e5d-4e16-8c8a-bd84956289cb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:19.886536 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:19.887493 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:19.887778 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:19.888174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:19.897821 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:19.899291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:59:19.912172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:19.913244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:59:19.915134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:19.915782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:19.915782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:19.916042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 523/1040] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:59:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:19.918021 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:19.918319 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:19.919823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:19.931731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:19.937146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:19.939636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:19.940716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 518/1041] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:19.944464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:19.963558 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.055654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b2238-777c-4d74-a32e-1129aac928c6 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.067901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.071103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c99cb-2dba-4a59-a558-f3552fad65bf None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.094928 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.095559 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.095842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.096110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.096880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:20.097190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.098710 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.112789 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.113429 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.113740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.114023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.114722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:20.115034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.116497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.125221 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 524/1042] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:59:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:20.127161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:20.130765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 519/1043] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:59:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:20.136583 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:20.226532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.251086 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13367638-df88-436a-91ec-e0a12b254106 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.257430 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e2cba-9cea-494e-8cab-46e4cd0c83c2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.294505 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.296245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.296815 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.297359 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] PATH_INFO: `/v3/users/bc135493f4ce4717b2e5e98aba699b18` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.297893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.298544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.298797 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.299075 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.316375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] RBAC: Authorizing `identity:get_user(user_id=bc135493f4ce4717b2e5e98aba699b18)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.317876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.320065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 520/1044] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/users/bc135493f4ce4717b2e5e98aba699b18 => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:20.328859 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:20.369877 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.404566 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.406506 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.447647 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef1346-a957-4e98-8d52-aaa144ecba83 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.484026 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.484633 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.484908 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.485176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.488646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.524953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 525/1045] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 11:59:20.530881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:20.561495 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.563159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.600212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.646306 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f796d-f5bd-44d0-a37d-f86cf12ecaf7 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.685085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 521/1046] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 11:59:20.691064 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:20.694206 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.694936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.695284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.695563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.708618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.710129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:20.727857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:20.728921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:20.729923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:20.729923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:20.729923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:20.733827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 526/1047] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:20.740562 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:20.748626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.850419 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52075f48-4773-461e-baef-b572b5eb3ce6 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.861634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97644474-b706-4433-bad6-3f051c6d8ec2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:20.872016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:20.887147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.887757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.888043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.888343 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.889541 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:20.890005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.891368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.903335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 527/1048] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:20.909033 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:20.919080 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:20.919736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:20.920021 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:20.920297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:20.922449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=39bec29c11d640cba8779e542a88d4a7)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:20.924423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:20.931579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 522/1049] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7 => generated 311 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:20.932650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:21.036553 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cdd59-c44a-4812-8aae-70dae0d8b693 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:21.052509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2967e-c51d-4969-8647-38688e223eb4 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:21.055464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:21.074018 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:21.074664 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:21.074958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:21.075228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:21.083097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:21.084687 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:59:21.097074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:21.098099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:21.099011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 528/1050] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:21.107476 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:21.111575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:21.112198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:21.112478 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:21.112750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:21.113482 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=39bec29c11d640cba8779e542a88d4a7, user.id=bc135493f4ce4717b2e5e98aba699b18) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:21.113819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:21.115263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:21.121225 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 523/1051] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:20 2022] GET /identity/v3/role_assignments?user.id=bc135493f4ce4717b2e5e98aba699b18&scope.project.id=39bec29c11d640cba8779e542a88d4a7&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:21.158140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:21.229819 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581f1cf-d742-464f-8dcd-02b19b820a85 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:21.272925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731370e-079c-4855-bdca-7c4eaa2f66f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:21.273798 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731370e-079c-4855-bdca-7c4eaa2f66f5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:21.273895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731370e-079c-4855-bdca-7c4eaa2f66f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:21.274114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731370e-079c-4855-bdca-7c4eaa2f66f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:21.274839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731370e-079c-4855-bdca-7c4eaa2f66f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:21.275146 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731370e-079c-4855-bdca-7c4eaa2f66f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:21.276628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731370e-079c-4855-bdca-7c4eaa2f66f5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:21.291477 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 529/1052] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:59:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:21.298272 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:21.425485 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50aec8-8685-4aee-b861-8eec302e59e8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:21.465945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58408-6929-4677-b619-b584adac8994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:21.466901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58408-6929-4677-b619-b584adac8994 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:21.467360 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58408-6929-4677-b619-b584adac8994 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:21.467805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58408-6929-4677-b619-b584adac8994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:21.468710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58408-6929-4677-b619-b584adac8994 None admin] RBAC: Adding query filter params (role.id=25ca9df14c1447128eda0c9db881eb51, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=0c8225879a66415c8a1dc28584e4c626) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:21.469224 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58408-6929-4677-b619-b584adac8994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:21.470958 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea58408-6929-4677-b619-b584adac8994 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:21.481902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 524/1053] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:21 2022] GET /identity/v3/role_assignments?user.id=0c8225879a66415c8a1dc28584e4c626&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=25ca9df14c1447128eda0c9db881eb51 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:22.223120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f0eb-2382-432b-89f5-b4e4423c262c None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:22.223396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f0eb-2382-432b-89f5-b4e4423c262c None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:22.223833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f0eb-2382-432b-89f5-b4e4423c262c None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:22.225213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 530/1054] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:22.234959 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d4fe-0918-4681-9d4f-10b9e4e14f92 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:22.235528 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d4fe-0918-4681-9d4f-10b9e4e14f92 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:22.235970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d4fe-0918-4681-9d4f-10b9e4e14f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:22.268184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a684d4fe-0918-4681-9d4f-10b9e4e14f92 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:22.270258 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a684d4fe-0918-4681-9d4f-10b9e4e14f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:22.309330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:22.368331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 525/1055] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:22.656741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f0eb-2382-432b-89f5-b4e4423c262c None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:22.842884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:22.850297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f0eb-2382-432b-89f5-b4e4423c262c None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:22.916228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4b046-51f2-4e42-91f9-312f26d4d1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:22.917212 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b046-51f2-4e42-91f9-312f26d4d1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:22.917731 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b046-51f2-4e42-91f9-312f26d4d1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:22.918304 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b046-51f2-4e42-91f9-312f26d4d1b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:22.919510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4b046-51f2-4e42-91f9-312f26d4d1b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:22.922088 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4b046-51f2-4e42-91f9-312f26d4d1b9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-9ee4b046-51f2-4e42-91f9-312f26d4d1b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:22.940569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:22.942374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:22.953252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 531/1056] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Sep 1 11:59:22 2022] GET /identity/v3/services/image => generated 86 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:22.955544 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d4fe-0918-4681-9d4f-10b9e4e14f92 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:23.059244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38cebc-eff8-416d-b955-b04fcdd4662d None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.059581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38cebc-eff8-416d-b955-b04fcdd4662d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.060044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38cebc-eff8-416d-b955-b04fcdd4662d None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.061898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 532/1057] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:23.068684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b175a-d55c-402b-b446-b4032a2049bc None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.069166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b175a-d55c-402b-b446-b4032a2049bc None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.069660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b175a-d55c-402b-b446-b4032a2049bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.093228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-123b175a-d55c-402b-b446-b4032a2049bc None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:23.094501 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-123b175a-d55c-402b-b446-b4032a2049bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:23.120655 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d4fe-0918-4681-9d4f-10b9e4e14f92 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:23.167220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:23.168030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.168527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.169018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.170322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] RBAC: Adding query filter params (name=image) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:23.170876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:23.173242 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:23.182184 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 526/1058] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:23.186357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 533/1059] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:23.191001 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:23.220491 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:23.349938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10572270-fabf-4862-ba65-33c54316ef9f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:23.398244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb287cd-2e7b-468b-8c45-dbc09344e0dd None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.398545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb287cd-2e7b-468b-8c45-dbc09344e0dd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.398795 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb287cd-2e7b-468b-8c45-dbc09344e0dd None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.400166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 534/1060] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:23.405760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:23.406313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d0d5-08ce-4a7d-956d-88482a76e426 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.406594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d0d5-08ce-4a7d-956d-88482a76e426 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.406863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d0d5-08ce-4a7d-956d-88482a76e426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.407415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.407837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.408487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.409531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:23.410273 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:23.412638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:23.424037 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 527/1061] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:23.426739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:23.432077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-a2f4d0d5-08ce-4a7d-956d-88482a76e426 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:23.436783 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-a2f4d0d5-08ce-4a7d-956d-88482a76e426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:23.536239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:23.538356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 535/1062] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:23.539245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d0d5-08ce-4a7d-956d-88482a76e426 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:23.550777 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b568a-e66d-4d7c-8372-d37df60456a1 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:23.567919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:23.597508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:23.598245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.598524 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.598766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.599413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:23.600794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:23.666061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d0d5-08ce-4a7d-956d-88482a76e426 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:23.678842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 528/1063] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 11:59:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:59:23.740976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:23.742209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.742708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.743122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.761239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:23.768271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:23.771070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:23.772955 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 536/1064] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:23.777506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:23.813940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:23.932418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d90f80-eee4-4abf-9b5c-b8e863cc57f8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:23.960049 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:23.982071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:23.983088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:23.983568 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:23.984055 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:23.985241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:23.985806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:23.988062 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a12649-84f1-4c1e-b7ef-04f0f4ef5b23 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:23.988311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:24.005169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 529/1065] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:24.011830 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:24.021570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:24.037951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:24.038628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:24.038914 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:24.039199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:24.052844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:24.058275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:24.059573 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:24.061209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 537/1066] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Sep 1 11:59:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:24.071688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:24.153013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf06706-d838-4b88-99a0-397a88ec65fb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:24.204206 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:24.205068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:24.205536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:24.205989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] PATH_INFO: `/v3/users/bc135493f4ce4717b2e5e98aba699b18` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:24.207459 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] RBAC: Authorizing `identity:get_user(user_id=bc135493f4ce4717b2e5e98aba699b18)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:24.209287 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:24.211345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 530/1067] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:24 2022] GET /identity/v3/users/bc135493f4ce4717b2e5e98aba699b18 => generated 285 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:24.218269 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:24.231896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19461bc-ec78-4674-9a56-452ae9edb6ae None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:24.296765 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:24.297741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:24.298191 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:24.298606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:24.299711 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:24.300192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:24.302595 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:24.313124 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:24.322258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 538/1068] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Sep 1 11:59:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:24.324972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:24.329904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:24.351986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7bd4a-2cd5-48c0-8671-10bbd45ba138 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:24.411331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:24.412534 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:24.412991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:24.413448 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:24.415316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] RBAC: Authorizing `identity:get_project(project_id=39bec29c11d640cba8779e542a88d4a7)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:24.418041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:24.422376 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 531/1069] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:24 2022] GET /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7 => generated 311 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:24.431362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:24.486165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d9296-e7e0-4411-acdd-2f39db17730b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:24.545495 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:24.546767 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:24.547255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:24.547745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:24.565163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:24.566666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:24.579137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354df21-d533-4cae-b07a-bf4f21e30185 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:24.585272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:24.586523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:24.587579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:24.587579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:24.587579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:24.587579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 539/1070] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:24.595701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:24.642945 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:24.644607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:24.644962 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:24.645327 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7/users/bc135493f4ce4717b2e5e98aba699b18/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:24.650240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] RBAC: Authorizing `identity:create_grant(project_id=39bec29c11d640cba8779e542a88d4a7, user_id=bc135493f4ce4717b2e5e98aba699b18, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:24.654483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:24.662462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:24.720911 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d909589-efa1-4d13-9a05-79fc8fee0a79 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:24.762790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:24.763550 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:24.764014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:24.764483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:24.765794 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:24.766316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:24.768674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:24.773912 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 532/1071] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:59:24 2022] PUT /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7/users/bc135493f4ce4717b2e5e98aba699b18/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:59:24.789895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 540/1072] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:24.792960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:24.798716 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:24.972059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bb73b-ecee-4963-8f42-a905539517eb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:25.007722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:25.008362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:25.008643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:25.008915 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:25.017991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:25.019371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:59:25.032117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:25.033350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:25.039276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 533/1073] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:59:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:25.039994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:25.176681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0460d-3a79-4c84-a0b0-5af0b8e50795 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:25.203628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:25.237189 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:25.238554 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:25.239376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:25.239985 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:25.241911 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:25.242692 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:25.245402 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:25.277902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 541/1074] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:59:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:25.284442 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:25.367341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:25.432953 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79654e-55de-481e-9d1e-b02fbf2ed923 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:25.492121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b7415-2efc-402d-9c3e-9e563912d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:25.493211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b7415-2efc-402d-9c3e-9e563912d015 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:25.493508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b7415-2efc-402d-9c3e-9e563912d015 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:25.493824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b7415-2efc-402d-9c3e-9e563912d015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:25.513196 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b7415-2efc-402d-9c3e-9e563912d015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:25.514735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b7415-2efc-402d-9c3e-9e563912d015 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-392b7415-2efc-402d-9c3e-9e563912d015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:25.531494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:25.540838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:25.542595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:25.542595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:25.542595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:25.542595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 534/1075] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:25.545484 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:25.683173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421eb27d-4a12-4265-8193-d940836e7fcd None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:25.733974 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:25.735003 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85758d-8dee-4e25-b013-af52e180bf0f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:25.735003 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85758d-8dee-4e25-b013-af52e180bf0f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:25.735285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:25.735285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:25.735559 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85758d-8dee-4e25-b013-af52e180bf0f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:25.735709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:25.735847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:25.736347 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:25.736695 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 535/1076] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:25.737575 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:25.747585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8480aa-7394-4c13-874c-0182d78b1e6f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:25.747585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8480aa-7394-4c13-874c-0182d78b1e6f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:25.747585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8480aa-7394-4c13-874c-0182d78b1e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:25.757496 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 542/1077] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:25.760965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:25.772509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0e8480aa-7394-4c13-874c-0182d78b1e6f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:25.773910 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0e8480aa-7394-4c13-874c-0182d78b1e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:25.807581 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:25.860545 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 536/1078] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:25.899369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad5d42-1410-472f-8711-ac754423fab5 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:25.940868 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:25.941781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:25.942213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:25.942635 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:25.951792 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:25.953521 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:59:25.967108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:25.968876 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:25.975020 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 543/1079] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:25.977113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8480aa-7394-4c13-874c-0182d78b1e6f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:26.031086 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:26.105712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8480aa-7394-4c13-874c-0182d78b1e6f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:26.136937 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:26.147949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:26.149393 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:26.149917 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:26.150715 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:26.151922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:26.152446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:26.155518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:26.177416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 537/1080] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:26.179351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:26.261285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:26.275304 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eee48-bb85-4d61-bd18-f874e95f9ae8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:26.322396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bad23-3160-4ca9-8cb7-2441ac86f8c0 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:26.326416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:26.342653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbde45-14a4-4c29-8558-3750b8a6c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:26.343277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbde45-14a4-4c29-8558-3750b8a6c82a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:26.343818 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbde45-14a4-4c29-8558-3750b8a6c82a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:26.343818 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbde45-14a4-4c29-8558-3750b8a6c82a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:26.384659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:26.385384 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:26.385805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:26.386301 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/0c8225879a66415c8a1dc28584e4c626/roles/25ca9df14c1447128eda0c9db881eb51` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:26.390472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=0c8225879a66415c8a1dc28584e4c626, role_id=25ca9df14c1447128eda0c9db881eb51)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:26.393028 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:26.423052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbde45-14a4-4c29-8558-3750b8a6c82a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:26.426620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbde45-14a4-4c29-8558-3750b8a6c82a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:26.459425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 538/1081] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:59:26 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/0c8225879a66415c8a1dc28584e4c626/roles/25ca9df14c1447128eda0c9db881eb51 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 11:59:26.602885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:26.627343 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 544/1082] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 11:59:26.636939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:26.821151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2de8b-48d3-4c8d-adc6-069d2c775311 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:26.827689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85840-bb07-484f-b048-c4089eadf14e None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:26.828217 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85840-bb07-484f-b048-c4089eadf14e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:26.828217 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85840-bb07-484f-b048-c4089eadf14e None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:26.829908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 545/1083] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:26.835604 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75b251-44d1-45b5-8b09-96be910d70f2 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:26.835866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75b251-44d1-45b5-8b09-96be910d70f2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:26.836152 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75b251-44d1-45b5-8b09-96be910d70f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:26.856057 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-5d75b251-44d1-45b5-8b09-96be910d70f2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:26.857361 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-5d75b251-44d1-45b5-8b09-96be910d70f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:26.880331 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:26.885398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:26.885398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:26.885398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:26.896963 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:26.899506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:26.925867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:26.926922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:26.927882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:26.927882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:26.927882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:26.928059 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 539/1084] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:26.934285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:26.949657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 546/1085] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:26.965887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:27.059539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cb2f6-68be-444c-8285-63443d90d413 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:27.099100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:27.099733 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:27.100017 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:27.100293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:27.100948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:27.101249 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:27.102689 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:27.121648 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 540/1086] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:27.230370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75b251-44d1-45b5-8b09-96be910d70f2 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:27.359163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75b251-44d1-45b5-8b09-96be910d70f2 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:27.386353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:27.401471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:27.402156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:27.402521 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:27.402801 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:27.414870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:27.419999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:27.420994 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:27.421901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 547/1087] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:27.428813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:27.530524 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:27.554442 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd77508-12d1-472f-acf9-248b1809fd3f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:27.594261 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:27.594871 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:27.595147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:27.595414 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:27.596076 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:27.596373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:27.597767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:27.606494 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 541/1088] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:27.612559 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:27.719059 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332de5c-b3d4-4e4e-b470-09d366c13e38 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:27.758097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:27.758738 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:27.759019 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:27.759284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] PATH_INFO: `/v3/users/bc135493f4ce4717b2e5e98aba699b18` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:27.760927 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] RBAC: Authorizing `identity:get_user(user_id=bc135493f4ce4717b2e5e98aba699b18)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:27.762385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:27.764805 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 548/1089] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:27 2022] GET /identity/v3/users/bc135493f4ce4717b2e5e98aba699b18 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:27.771131 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:27.876968 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3bcb2-86b5-4677-9389-8d315afe56eb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:27.911155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:27.911771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:27.912055 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:27.912326 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:27.921839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:27.982788 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:27.984254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:28.097964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 542/1090] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 11:59:28.104381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:28.159008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:28.220372 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fb45e-e0a2-4dbb-8fca-83345b5b6a7a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:28.259879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadcc83-2bbe-4fba-bb64-24763eb94e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:28.260532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadcc83-2bbe-4fba-bb64-24763eb94e8f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:28.260853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadcc83-2bbe-4fba-bb64-24763eb94e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:28.261135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadcc83-2bbe-4fba-bb64-24763eb94e8f None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:28.262399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadcc83-2bbe-4fba-bb64-24763eb94e8f None admin] RBAC: Authorizing `identity:get_project(project_id=39bec29c11d640cba8779e542a88d4a7)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:28.263939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadcc83-2bbe-4fba-bb64-24763eb94e8f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:28.265744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 549/1091] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:28 2022] GET /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:28.272338 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:28.321299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:28.419238 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93706138-5d3e-4af3-91ff-a052d2b167ec None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:28.421219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ec0c6-1c7c-493f-9a54-a4acc8cf4410 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:28.421511 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ec0c6-1c7c-493f-9a54-a4acc8cf4410 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:28.421838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ec0c6-1c7c-493f-9a54-a4acc8cf4410 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:28.423545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 550/1092] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 11:59:28.431182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5e8e-f742-49b8-9707-7faf73567f71 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:28.431464 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5e8e-f742-49b8-9707-7faf73567f71 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:28.431741 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5e8e-f742-49b8-9707-7faf73567f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:28.451666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-fd7f5e8e-f742-49b8-9707-7faf73567f71 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:28.452883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-fd7f5e8e-f742-49b8-9707-7faf73567f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:28.466088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:28.466795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:28.467077 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:28.467352 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:28.468089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=39bec29c11d640cba8779e542a88d4a7, user.id=bc135493f4ce4717b2e5e98aba699b18) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:28.468405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:28.469881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:28.476189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 543/1093] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:28 2022] GET /identity/v3/role_assignments?user.id=bc135493f4ce4717b2e5e98aba699b18&scope.project.id=39bec29c11d640cba8779e542a88d4a7&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:28.530920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 551/1094] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:28.797311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:28.901933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1239ee5-eee9-4343-91e4-e8d52a7d2cee None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:28.903532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:28.938847 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:28.939472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:28.939750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:28.940021 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:28.951713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:28.956548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:28.957564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:28.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 544/1095] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Sep 1 11:59:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:28.963827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5e8e-f742-49b8-9707-7faf73567f71 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:29.017532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:29.077247 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5e8e-f742-49b8-9707-7faf73567f71 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:29.114162 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:29.114915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:29.115040 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:29.115315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:29.115988 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:29.116297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:29.117693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:29.127127 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 552/1096] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Sep 1 11:59:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:29.132947 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:29.240134 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53801c97-2ac9-45b9-80dd-e969299dd8bd None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:29.276169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:29.276789 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:29.277066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:29.277066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:29.290751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:29.292236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:29.307725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:29.308881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:29.310825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:29.310825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:29.310825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:29.310825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 545/1097] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:29.315520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:29.427698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7a5f2-fe78-47f8-b013-428dd88eafea None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:29.438637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:29.463776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:29.464388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:29.464668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:29.464942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:29.465587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:29.465920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:29.467376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:29.487096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 553/1098] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:29.488865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:29.508100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:29.599031 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3028bb-c52c-43be-98bb-157557f604bb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:29.635799 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:29.636439 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:29.636710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:29.636981 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:29.645804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:29.647303 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 11:59:29.659660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:29.660688 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:29.662178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 546/1099] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Sep 1 11:59:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:29.667056 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:29.780527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce571-93f4-4593-bae7-b4082015e6cc None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:29.815779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:29.816393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:29.816665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:29.816932 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:29.817703 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:29.818011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:29.819695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:29.838838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 554/1100] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Sep 1 11:59:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:29.845106 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:29.924796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:29.951580 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53014063-c11f-4d6f-8070-351322d56941 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:29.987489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:29.988107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:29.988381 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:29.988778 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.059299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:30.060715 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:30.136714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:30.169715 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 547/1101] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:59:30.177249 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:30.209622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:30.299161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8197f1-f1a8-422c-b3e3-24c6d570f3e3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:30.337299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cf962-37fa-45f8-95aa-77a3dc209f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:30.337890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf962-37fa-45f8-95aa-77a3dc209f1a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:30.338170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf962-37fa-45f8-95aa-77a3dc209f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:30.338438 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf962-37fa-45f8-95aa-77a3dc209f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.349585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cf962-37fa-45f8-95aa-77a3dc209f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:30.351060 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cf962-37fa-45f8-95aa-77a3dc209f1a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-c45cf962-37fa-45f8-95aa-77a3dc209f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:30.365745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:30.371648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 11:59:30.372823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:30.372823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 11:59:30.372823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:30.372823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 555/1102] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:30.373131 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:30.447584 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8660-17ef-47b2-a0af-c27de6273333 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:30.447930 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8660-17ef-47b2-a0af-c27de6273333 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:30.448147 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8660-17ef-47b2-a0af-c27de6273333 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.449575 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 556/1103] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:30.455461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46ab87-ecf1-48f0-b23d-efc3aabf9572 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:30.455748 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46ab87-ecf1-48f0-b23d-efc3aabf9572 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:30.456028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46ab87-ecf1-48f0-b23d-efc3aabf9572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.473991 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-1c46ab87-ecf1-48f0-b23d-efc3aabf9572 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:30.474440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259a76c-1b44-473f-affb-6d496bd61e64 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:30.475187 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-1c46ab87-ecf1-48f0-b23d-efc3aabf9572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:30.508930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:30.509548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:30.509862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:30.510139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.510785 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:30.511095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:30.512501 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:30.526129 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 548/1104] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 11:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:30.531883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:30.543496 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 557/1105] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 11:59:30.570193 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:30.639466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d7f24-aa40-44e5-bcbb-b2ae4bcb4e8f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:30.676004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:30.676638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:30.676917 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:30.677192 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.685138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:30.686618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 11:59:30.698861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:30.705197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:30.706248 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 549/1106] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 11:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 11:59:30.706756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46ab87-ecf1-48f0-b23d-efc3aabf9572 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:30.719936 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:30.813421 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:30.817635 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46ab87-ecf1-48f0-b23d-efc3aabf9572 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:30.854578 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:30.855199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:30.855480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:30.855751 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.856588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:30.856764 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:30.858286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:30.872284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 558/1107] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 11:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:30.878197 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:30.922581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:30.929096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4b1ec-da8f-45e1-9cfe-94093cfc57aa None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:30.950025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:30.965018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:30.965243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:30.965363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:30.965713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:30.967394 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:30.968835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:30.970512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 550/1108] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:30.976454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:30.986740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd8e1-c876-48ab-80b5-6cb2318b5e19 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:31.023587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5dc50-8c97-4608-9caa-cd53abfafd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:31.024258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5dc50-8c97-4608-9caa-cd53abfafd6f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:31.024657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5dc50-8c97-4608-9caa-cd53abfafd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:31.024939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5dc50-8c97-4608-9caa-cd53abfafd6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:31.025713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5dc50-8c97-4608-9caa-cd53abfafd6f None admin] RBAC: Adding query filter params (role.id=25ca9df14c1447128eda0c9db881eb51, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=0c8225879a66415c8a1dc28584e4c626) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:31.026122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5dc50-8c97-4608-9caa-cd53abfafd6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:31.027858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5dc50-8c97-4608-9caa-cd53abfafd6f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:31.035596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 559/1109] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:30 2022] GET /identity/v3/role_assignments?user.id=0c8225879a66415c8a1dc28584e4c626&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=25ca9df14c1447128eda0c9db881eb51 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:31.086529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654669-bb76-4930-bf96-be66bafd8a38 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:31.128664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e8f73-35c0-4787-a599-2f7711be5ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:31.129299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e8f73-35c0-4787-a599-2f7711be5ead None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:31.129578 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e8f73-35c0-4787-a599-2f7711be5ead None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:31.129881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e8f73-35c0-4787-a599-2f7711be5ead None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:31.130584 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e8f73-35c0-4787-a599-2f7711be5ead None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:31.132052 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e8f73-35c0-4787-a599-2f7711be5ead None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:31.229539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 551/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:59:30 2022] POST /identity/v3/users => generated 286 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 11:59:33.211848 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b231d-5cb2-4513-b38a-d8962580225c None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:33.212383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b231d-5cb2-4513-b38a-d8962580225c None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:33.212383 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b231d-5cb2-4513-b38a-d8962580225c None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:33.213788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 560/1111] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:33.220333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37586-a20f-4391-b37f-6df889f83f0a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:33.220790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37586-a20f-4391-b37f-6df889f83f0a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:33.221010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37586-a20f-4391-b37f-6df889f83f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:33.239717 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c7c37586-a20f-4391-b37f-6df889f83f0a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:33.240938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c7c37586-a20f-4391-b37f-6df889f83f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:33.254981 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:33.310583 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 552/1112] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:33.581532 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b231d-5cb2-4513-b38a-d8962580225c None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:33.681744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b231d-5cb2-4513-b38a-d8962580225c None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:33.716998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:33.717657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:33.717944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:33.718222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] PATH_INFO: `/v3/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:33.719640 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] RBAC: Authorizing `identity:get_role(role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:33.721014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:33.722385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 561/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:33 2022] GET /identity/v3/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:33.728847 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c37586-a20f-4391-b37f-6df889f83f0a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:33.842035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c37586-a20f-4391-b37f-6df889f83f0a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:33.877533 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:33.878201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:33.878481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:33.878754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] PATH_INFO: `/v3/users/0be688391f2d4b8ab2933c31a2cf1639` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:33.893373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] RBAC: Authorizing `identity:get_user(user_id=0be688391f2d4b8ab2933c31a2cf1639)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:33.898719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:33.898719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 553/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:33 2022] GET /identity/v3/users/0be688391f2d4b8ab2933c31a2cf1639 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:33.902624 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:33.905644 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:34.010538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853db0d-39fe-4605-be6f-eceb226dd63e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:34.049316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:34.049981 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:34.050263 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:34.050531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:34.121488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:34.123585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:34.125654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:34.229986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 562/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:59:34.236748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:34.247273 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:34.343895 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff429d-bf27-44b4-834a-24f0cbc09a95 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:34.382826 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bde4b-1546-4a5f-b829-afd70c567ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:34.383371 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bde4b-1546-4a5f-b829-afd70c567ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:34.383649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bde4b-1546-4a5f-b829-afd70c567ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:34.383923 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bde4b-1546-4a5f-b829-afd70c567ad1 None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:34.385060 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bde4b-1546-4a5f-b829-afd70c567ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=39bec29c11d640cba8779e542a88d4a7)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:34.386535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bde4b-1546-4a5f-b829-afd70c567ad1 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:34.388239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 554/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:34 2022] GET /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:34.394568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:34.543110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e955a-ebfb-4901-8f0e-c57f8d8d848f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:34.545219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:34.580199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944894e-fd26-424d-b34c-355e4c00c1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:34.580825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944894e-fd26-424d-b34c-355e4c00c1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:34.581106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944894e-fd26-424d-b34c-355e4c00c1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:34.581379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944894e-fd26-424d-b34c-355e4c00c1a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:34.582174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944894e-fd26-424d-b34c-355e4c00c1a4 None admin] RBAC: Adding query filter params (role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=39bec29c11d640cba8779e542a88d4a7, user.id=0be688391f2d4b8ab2933c31a2cf1639) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:34.582488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944894e-fd26-424d-b34c-355e4c00c1a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:34.583967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944894e-fd26-424d-b34c-355e4c00c1a4 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:34.590351 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 563/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:34 2022] GET /identity/v3/role_assignments?user.id=0be688391f2d4b8ab2933c31a2cf1639&scope.project.id=39bec29c11d640cba8779e542a88d4a7&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:36.521733 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a17e1-5436-4a19-8f88-7bdddb598546 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:36.522243 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a17e1-5436-4a19-8f88-7bdddb598546 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:36.522332 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a17e1-5436-4a19-8f88-7bdddb598546 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:36.523623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 555/1118] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:36.528796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce169f-97b3-4838-ad54-616bc9c39a69 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:36.529170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce169f-97b3-4838-ad54-616bc9c39a69 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:36.529357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce169f-97b3-4838-ad54-616bc9c39a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:36.547223 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-57ce169f-97b3-4838-ad54-616bc9c39a69 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:36.548417 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-57ce169f-97b3-4838-ad54-616bc9c39a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:36.614389 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 564/1119] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:36.885246 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a17e1-5436-4a19-8f88-7bdddb598546 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:36.960106 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:36.985007 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a17e1-5436-4a19-8f88-7bdddb598546 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:37.018405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:37.019043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:37.019305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:37.019579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] PATH_INFO: `/v3/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:37.020960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] RBAC: Authorizing `identity:get_role(role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:37.022352 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:37.023669 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 556/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:36 2022] GET /identity/v3/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:37.029543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce169f-97b3-4838-ad54-616bc9c39a69 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:37.120363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:37.126583 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce169f-97b3-4838-ad54-616bc9c39a69 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:37.160173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:37.160725 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:37.161008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:37.161285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] PATH_INFO: `/v3/users/0be688391f2d4b8ab2933c31a2cf1639` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:37.162443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=0be688391f2d4b8ab2933c31a2cf1639)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:37.163878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:37.165456 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 565/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:37 2022] GET /identity/v3/users/0be688391f2d4b8ab2933c31a2cf1639 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:37.172091 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:37.271355 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5565cb-ab23-4bf0-9bc0-85be46a6bc72 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:37.304154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a7b03-71aa-488d-9f61-1982b24b0490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:37.304779 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a7b03-71aa-488d-9f61-1982b24b0490 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:37.305057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a7b03-71aa-488d-9f61-1982b24b0490 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:37.305325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a7b03-71aa-488d-9f61-1982b24b0490 None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:37.306455 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a7b03-71aa-488d-9f61-1982b24b0490 None admin] RBAC: Authorizing `identity:get_project(project_id=39bec29c11d640cba8779e542a88d4a7)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:37.307877 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a7b03-71aa-488d-9f61-1982b24b0490 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:37.309489 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 557/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:37 2022] GET /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:37.316776 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:37.415121 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1354ca-afb0-400c-9d6e-34cc9a51f9a3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:37.449306 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a975f68-50ba-46f9-9798-60ddce82fadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:37.450069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a975f68-50ba-46f9-9798-60ddce82fadc None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:37.450264 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a975f68-50ba-46f9-9798-60ddce82fadc None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:37.451067 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a975f68-50ba-46f9-9798-60ddce82fadc None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7/users/0be688391f2d4b8ab2933c31a2cf1639/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:37.453199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a975f68-50ba-46f9-9798-60ddce82fadc None admin] RBAC: Authorizing `identity:create_grant(project_id=39bec29c11d640cba8779e542a88d4a7, user_id=0be688391f2d4b8ab2933c31a2cf1639, role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:37.454657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a975f68-50ba-46f9-9798-60ddce82fadc None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:37.514413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 566/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:59:37 2022] PUT /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7/users/0be688391f2d4b8ab2933c31a2cf1639/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:59:39.398621 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda54396-13c5-4cb3-a8ec-f74df84f9c8f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:39.399181 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda54396-13c5-4cb3-a8ec-f74df84f9c8f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:39.399181 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda54396-13c5-4cb3-a8ec-f74df84f9c8f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:39.400522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 558/1124] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:39.406386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29139a-e065-4ff7-95c3-ed8343ed11e0 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:39.406793 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29139a-e065-4ff7-95c3-ed8343ed11e0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:39.406952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29139a-e065-4ff7-95c3-ed8343ed11e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:39.425524 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-fe29139a-e065-4ff7-95c3-ed8343ed11e0 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:39.426756 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-fe29139a-e065-4ff7-95c3-ed8343ed11e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:39.443907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:39.493327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 567/1125] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:39.761564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda54396-13c5-4cb3-a8ec-f74df84f9c8f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:39.773838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:39.861575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda54396-13c5-4cb3-a8ec-f74df84f9c8f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:39.894875 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:39.895496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:39.895771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:39.896035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] PATH_INFO: `/v3/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:39.897400 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] RBAC: Authorizing `identity:get_role(role_id=f8eb11a8a4ab4bd3a62b3c1ba2a97271)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:39.898791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:39.900110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 559/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:39 2022] GET /identity/v3/roles/f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:39.906950 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29139a-e065-4ff7-95c3-ed8343ed11e0 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:40.003377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29139a-e065-4ff7-95c3-ed8343ed11e0 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:40.037102 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:40.037675 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:40.037959 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:40.038230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] PATH_INFO: `/v3/users/0be688391f2d4b8ab2933c31a2cf1639` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:40.039338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] RBAC: Authorizing `identity:get_user(user_id=0be688391f2d4b8ab2933c31a2cf1639)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:40.040746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:40.042356 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 568/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:39 2022] GET /identity/v3/users/0be688391f2d4b8ab2933c31a2cf1639 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 11:59:40.049207 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:40.126784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:40.147661 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee2c2-d5ba-49cc-afac-63a37961b29f None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:40.183891 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:40.184499 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:40.184804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:40.185069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:40.251173 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:40.252625 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:40.328010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:40.350115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 560/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:59:40.356292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:40.381031 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:40.456530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2435af-426e-4c12-b9ef-682ed309c4b2 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:40.490234 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57493d-11b6-42b0-a6c6-f3d83f04bd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:40.490861 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57493d-11b6-42b0-a6c6-f3d83f04bd21 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:40.491139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57493d-11b6-42b0-a6c6-f3d83f04bd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:40.491409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57493d-11b6-42b0-a6c6-f3d83f04bd21 None admin] PATH_INFO: `/v3/projects/39bec29c11d640cba8779e542a88d4a7` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:40.492526 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57493d-11b6-42b0-a6c6-f3d83f04bd21 None admin] RBAC: Authorizing `identity:get_project(project_id=39bec29c11d640cba8779e542a88d4a7)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:40.493978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57493d-11b6-42b0-a6c6-f3d83f04bd21 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:40.495618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 569/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:40 2022] GET /identity/v3/projects/39bec29c11d640cba8779e542a88d4a7 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:40.501866 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:40.608829 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b454794-9b73-4327-b85e-3cabc28306c2 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:40.641985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e9845-6b9e-40db-b85c-1f9206c01ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:40.642590 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e9845-6b9e-40db-b85c-1f9206c01ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:40.642883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e9845-6b9e-40db-b85c-1f9206c01ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:40.643145 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e9845-6b9e-40db-b85c-1f9206c01ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:40.643874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e9845-6b9e-40db-b85c-1f9206c01ea7 None admin] RBAC: Adding query filter params (role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271, scope.project.id=39bec29c11d640cba8779e542a88d4a7, user.id=0be688391f2d4b8ab2933c31a2cf1639) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:40.644179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e9845-6b9e-40db-b85c-1f9206c01ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:40.645707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e9845-6b9e-40db-b85c-1f9206c01ea7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:40.651599 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 561/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:40 2022] GET /identity/v3/role_assignments?user.id=0be688391f2d4b8ab2933c31a2cf1639&scope.project.id=39bec29c11d640cba8779e542a88d4a7&role.id=f8eb11a8a4ab4bd3a62b3c1ba2a97271 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:42.571050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726808c-8d6c-429b-9fda-986ef9450ec0 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:42.571589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726808c-8d6c-429b-9fda-986ef9450ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:42.571589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726808c-8d6c-429b-9fda-986ef9450ec0 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:42.572933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 570/1131] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:42.578606 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e72726-9617-4bf6-b43e-225bc2263822 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:42.578989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e72726-9617-4bf6-b43e-225bc2263822 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:42.579151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e72726-9617-4bf6-b43e-225bc2263822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:42.596897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-55e72726-9617-4bf6-b43e-225bc2263822 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:42.598105 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-55e72726-9617-4bf6-b43e-225bc2263822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:42.632022 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:42.663497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 562/1132] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:42.938123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726808c-8d6c-429b-9fda-986ef9450ec0 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:43.027229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:43.038024 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726808c-8d6c-429b-9fda-986ef9450ec0 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:43.074351 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4f691-87c3-4087-8b1f-44fbe95bb6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:43.074981 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4f691-87c3-4087-8b1f-44fbe95bb6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:43.075261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4f691-87c3-4087-8b1f-44fbe95bb6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:43.075577 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4f691-87c3-4087-8b1f-44fbe95bb6b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:43.076718 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4f691-87c3-4087-8b1f-44fbe95bb6b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:43.078224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4f691-87c3-4087-8b1f-44fbe95bb6b9 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:43.079889 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 571/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:43.085857 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e72726-9617-4bf6-b43e-225bc2263822 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:43.184015 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e72726-9617-4bf6-b43e-225bc2263822 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:43.225941 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b71ee-6bc4-455d-8283-cd75585c772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:43.226509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b71ee-6bc4-455d-8283-cd75585c772b None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:43.226773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b71ee-6bc4-455d-8283-cd75585c772b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:43.227043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b71ee-6bc4-455d-8283-cd75585c772b None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:43.227689 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b71ee-6bc4-455d-8283-cd75585c772b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:43.229119 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b71ee-6bc4-455d-8283-cd75585c772b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:43.297811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 563/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 11:59:43 2022] POST /identity/v3/projects => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:45.267044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009406de-7b0d-4818-8308-594f5d714c6d None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:45.267635 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009406de-7b0d-4818-8308-594f5d714c6d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:45.267635 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009406de-7b0d-4818-8308-594f5d714c6d None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:45.268908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 572/1135] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:45.274155 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a609552-20dd-4a4e-bd6f-533000152df9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:45.274593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a609552-20dd-4a4e-bd6f-533000152df9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:45.274699 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a609552-20dd-4a4e-bd6f-533000152df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:45.293296 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0a609552-20dd-4a4e-bd6f-533000152df9 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:45.295023 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0a609552-20dd-4a4e-bd6f-533000152df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:45.312695 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:45.362880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 564/1136] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:45.634468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009406de-7b0d-4818-8308-594f5d714c6d None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:45.735269 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009406de-7b0d-4818-8308-594f5d714c6d None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:45.767258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ad6e1-ca4b-446d-bb96-324c87862d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:45.767871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad6e1-ca4b-446d-bb96-324c87862d77 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:45.768170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad6e1-ca4b-446d-bb96-324c87862d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:45.768439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad6e1-ca4b-446d-bb96-324c87862d77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:45.769529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ad6e1-ca4b-446d-bb96-324c87862d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:45.770970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ad6e1-ca4b-446d-bb96-324c87862d77 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:45.772508 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 573/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 11:59:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:45.778960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a609552-20dd-4a4e-bd6f-533000152df9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:45.875643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a609552-20dd-4a4e-bd6f-533000152df9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:45.908985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f744f8-8c65-4a5c-b4aa-d29ab66ad558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:45.909524 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f744f8-8c65-4a5c-b4aa-d29ab66ad558 None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:45.909829 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f744f8-8c65-4a5c-b4aa-d29ab66ad558 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:45.910104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f744f8-8c65-4a5c-b4aa-d29ab66ad558 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:45.910750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f744f8-8c65-4a5c-b4aa-d29ab66ad558 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:45.912180 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f744f8-8c65-4a5c-b4aa-d29ab66ad558 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:45.976553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 565/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 11:59:45 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 11:59:47.927736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b4f2b-6af0-4f9b-b64a-8ba865d4b5cb None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:47.928308 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b4f2b-6af0-4f9b-b64a-8ba865d4b5cb None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:47.928308 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b4f2b-6af0-4f9b-b64a-8ba865d4b5cb None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:47.929651 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 574/1139] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:47.935473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2f36-fb1b-452b-b058-b12bdf6ee05a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:47.935896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2f36-fb1b-452b-b058-b12bdf6ee05a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:47.936019 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2f36-fb1b-452b-b058-b12bdf6ee05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:47.955820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3a8c2f36-fb1b-452b-b058-b12bdf6ee05a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:47.957024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3a8c2f36-fb1b-452b-b058-b12bdf6ee05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:47.959860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:48.024452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 566/1140] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:48.294875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b4f2b-6af0-4f9b-b64a-8ba865d4b5cb None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:48.322482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:48.395804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b4f2b-6af0-4f9b-b64a-8ba865d4b5cb None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:48.429501 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:48.430166 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:48.430456 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:48.430735 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:48.440589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:48.445716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:48.446921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:48.447949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 575/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:48.452911 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c2f36-fb1b-452b-b058-b12bdf6ee05a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:48.549444 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c2f36-fb1b-452b-b058-b12bdf6ee05a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:48.582307 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:48.582906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:48.583179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:48.583449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:48.584095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:48.584389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:48.585776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:48.594252 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 567/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:48.600138 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:48.684522 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:48.699163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a79ca-34ec-41c3-b776-94f21f2db4fa None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:48.733806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:48.734441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:48.734725 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:48.735002 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] PATH_INFO: `/v3/users/1cddaabdc2cb4c6782cbf8b59262dbf8` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:48.748374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] RBAC: Authorizing `identity:get_user(user_id=1cddaabdc2cb4c6782cbf8b59262dbf8)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:48.749843 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:48.751469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 576/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:48 2022] GET /identity/v3/users/1cddaabdc2cb4c6782cbf8b59262dbf8 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:48.757531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:48.760446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:48.862286 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eae75-9eb5-4f9a-b119-8461defd016e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:48.897111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:48.897766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:48.898034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:48.898310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:48.964072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:48.965532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:48.967913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:49.062732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 568/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:59:49.068993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:49.169712 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235181d-41cb-4ff1-938c-d7314ffd5c13 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:49.205791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a23cf-9cb8-4835-b5f9-0cadec3c9335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:49.206426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a23cf-9cb8-4835-b5f9-0cadec3c9335 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:49.206714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a23cf-9cb8-4835-b5f9-0cadec3c9335 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:49.206990 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a23cf-9cb8-4835-b5f9-0cadec3c9335 None admin] PATH_INFO: `/v3/projects/6051a43122894939a366a52dd54b6f87` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:49.208116 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a23cf-9cb8-4835-b5f9-0cadec3c9335 None admin] RBAC: Authorizing `identity:get_project(project_id=6051a43122894939a366a52dd54b6f87)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:49.209537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a23cf-9cb8-4835-b5f9-0cadec3c9335 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:49.211215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 577/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:49 2022] GET /identity/v3/projects/6051a43122894939a366a52dd54b6f87 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:49.218006 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:49.318606 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45bcdb-505a-4d02-be6e-b54b292d433d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:49.320186 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:49.353475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc38d4c-5b97-4b3d-8af6-88011ca9daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:49.354104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38d4c-5b97-4b3d-8af6-88011ca9daaa None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:49.354377 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38d4c-5b97-4b3d-8af6-88011ca9daaa None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:49.354647 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38d4c-5b97-4b3d-8af6-88011ca9daaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:49.355362 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc38d4c-5b97-4b3d-8af6-88011ca9daaa None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=6051a43122894939a366a52dd54b6f87, user.id=1cddaabdc2cb4c6782cbf8b59262dbf8) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:49.355666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc38d4c-5b97-4b3d-8af6-88011ca9daaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:49.357707 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc38d4c-5b97-4b3d-8af6-88011ca9daaa None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:49.363223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 569/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:49 2022] GET /identity/v3/role_assignments?user.id=1cddaabdc2cb4c6782cbf8b59262dbf8&scope.project.id=6051a43122894939a366a52dd54b6f87&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:51.238008 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23568aba-f832-4cf7-8b50-8fca10d383cb None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:51.238551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23568aba-f832-4cf7-8b50-8fca10d383cb None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:51.238659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23568aba-f832-4cf7-8b50-8fca10d383cb None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:51.239920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 578/1147] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:51.245109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3015482-2494-4b00-9bb1-c19abcc9531e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:51.245560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3015482-2494-4b00-9bb1-c19abcc9531e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:51.245681 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3015482-2494-4b00-9bb1-c19abcc9531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:51.264316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e3015482-2494-4b00-9bb1-c19abcc9531e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:51.265499 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e3015482-2494-4b00-9bb1-c19abcc9531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:51.330906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 570/1148] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:51.605812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23568aba-f832-4cf7-8b50-8fca10d383cb None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:51.620921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:51.705864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23568aba-f832-4cf7-8b50-8fca10d383cb None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:51.740288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:51.740917 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:51.741196 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:51.741470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:51.751558 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:51.756385 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:51.757500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:51.758743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 579/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:59:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:51.763672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3015482-2494-4b00-9bb1-c19abcc9531e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:51.857169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:51.863334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3015482-2494-4b00-9bb1-c19abcc9531e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:51.896633 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:51.901368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:51.901368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:51.901368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:51.901368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:51.901368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:51.901368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:51.908683 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 571/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:59:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:51.914578 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:51.986461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:52.015739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab896b17-b657-47c0-bf67-39e32ebfd00a None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:52.050925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:52.051609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:52.051890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:52.052162 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] PATH_INFO: `/v3/users/1cddaabdc2cb4c6782cbf8b59262dbf8` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:52.053286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] RBAC: Authorizing `identity:get_user(user_id=1cddaabdc2cb4c6782cbf8b59262dbf8)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:52.054739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:52.056336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 580/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:51 2022] GET /identity/v3/users/1cddaabdc2cb4c6782cbf8b59262dbf8 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:52.062797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:52.167316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6596b-0e43-45df-81e5-c23c754b7114 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:52.199967 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05283b-cc1b-4261-b1bf-568e76d6a84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:52.200607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05283b-cc1b-4261-b1bf-568e76d6a84f None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:52.200879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05283b-cc1b-4261-b1bf-568e76d6a84f None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:52.201146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05283b-cc1b-4261-b1bf-568e76d6a84f None admin] PATH_INFO: `/v3/projects/6051a43122894939a366a52dd54b6f87` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:52.202254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05283b-cc1b-4261-b1bf-568e76d6a84f None admin] RBAC: Authorizing `identity:get_project(project_id=6051a43122894939a366a52dd54b6f87)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:52.203692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05283b-cc1b-4261-b1bf-568e76d6a84f None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:52.205288 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 572/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:52 2022] GET /identity/v3/projects/6051a43122894939a366a52dd54b6f87 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:52.211375 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:52.316932 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2952c-0cd8-49a8-b66e-755de12a7216 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:52.353152 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42f19a-9ce9-497d-b307-72d520bf9844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:52.353872 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42f19a-9ce9-497d-b307-72d520bf9844 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:52.354151 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42f19a-9ce9-497d-b307-72d520bf9844 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:52.354429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42f19a-9ce9-497d-b307-72d520bf9844 None admin] PATH_INFO: `/v3/projects/6051a43122894939a366a52dd54b6f87/users/1cddaabdc2cb4c6782cbf8b59262dbf8/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:52.356297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42f19a-9ce9-497d-b307-72d520bf9844 None admin] RBAC: Authorizing `identity:create_grant(project_id=6051a43122894939a366a52dd54b6f87, user_id=1cddaabdc2cb4c6782cbf8b59262dbf8, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:52.358307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42f19a-9ce9-497d-b307-72d520bf9844 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:52.418647 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:52.423941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 581/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 11:59:52 2022] PUT /identity/v3/projects/6051a43122894939a366a52dd54b6f87/users/1cddaabdc2cb4c6782cbf8b59262dbf8/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 11:59:54.305036 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bd565-3bef-41ef-8ad7-53db9491fb91 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:54.305597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bd565-3bef-41ef-8ad7-53db9491fb91 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:54.305597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bd565-3bef-41ef-8ad7-53db9491fb91 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:54.306954 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 573/1154] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:54.312390 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319afc0-3ea4-471e-a66e-0b6358411adc None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:54.312766 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319afc0-3ea4-471e-a66e-0b6358411adc None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:54.312952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319afc0-3ea4-471e-a66e-0b6358411adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:54.331683 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2319afc0-3ea4-471e-a66e-0b6358411adc None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:54.332885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2319afc0-3ea4-471e-a66e-0b6358411adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:54.397652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 582/1155] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:54.665643 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bd565-3bef-41ef-8ad7-53db9491fb91 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:54.694334 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:54.772100 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bd565-3bef-41ef-8ad7-53db9491fb91 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:54.805816 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:54.806445 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:54.806724 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:54.806996 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:54.818667 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 11:59:54.823497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:54.824497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 11:59:54.825417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 574/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 11:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:54.830929 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319afc0-3ea4-471e-a66e-0b6358411adc None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:54.931106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319afc0-3ea4-471e-a66e-0b6358411adc None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:54.959973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:54.965619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:54.966241 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:54.966520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:54.966797 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:54.967458 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:54.967763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:54.969123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:54.977157 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 583/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 11:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:54.983749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:55.069672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:55.083885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff5765-47d9-4626-a37e-595902b726bc None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:55.116588 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:55.117216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:55.117492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:55.117798 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] PATH_INFO: `/v3/users/1cddaabdc2cb4c6782cbf8b59262dbf8` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:55.118991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] RBAC: Authorizing `identity:get_user(user_id=1cddaabdc2cb4c6782cbf8b59262dbf8)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:55.120422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:55.122071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 575/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 11:59:54 2022] GET /identity/v3/users/1cddaabdc2cb4c6782cbf8b59262dbf8 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:55.131799 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:55.229284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe18f8-5e4c-4f54-b78d-c38b73d8e62b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:55.262704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:55.263307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:55.263584 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:55.263853 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:55.320941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:55.330171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:55.331624 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:55.428032 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 584/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:59:55.434194 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:55.532691 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e951e-6c40-4398-9828-0cb6ed8a6728 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:55.565597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d043e6-5fff-48b4-816a-f9c94eb32049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:55.566242 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d043e6-5fff-48b4-816a-f9c94eb32049 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:55.566518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d043e6-5fff-48b4-816a-f9c94eb32049 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:55.566785 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d043e6-5fff-48b4-816a-f9c94eb32049 None admin] PATH_INFO: `/v3/projects/6051a43122894939a366a52dd54b6f87` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:55.567895 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d043e6-5fff-48b4-816a-f9c94eb32049 None admin] RBAC: Authorizing `identity:get_project(project_id=6051a43122894939a366a52dd54b6f87)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:55.569573 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d043e6-5fff-48b4-816a-f9c94eb32049 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:55.571214 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 576/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 11:59:55 2022] GET /identity/v3/projects/6051a43122894939a366a52dd54b6f87 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:55.578100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:55.685839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:55.694652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c57df-7f99-47c2-ba6e-d5a1adc4c761 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:55.730001 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d26636-e2b6-434d-9ea2-b2966bd82b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:55.730618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d26636-e2b6-434d-9ea2-b2966bd82b55 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:55.730895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d26636-e2b6-434d-9ea2-b2966bd82b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:55.731164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d26636-e2b6-434d-9ea2-b2966bd82b55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:55.732656 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d26636-e2b6-434d-9ea2-b2966bd82b55 None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=6051a43122894939a366a52dd54b6f87, user.id=1cddaabdc2cb4c6782cbf8b59262dbf8) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:55.732962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d26636-e2b6-434d-9ea2-b2966bd82b55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:55.734404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d26636-e2b6-434d-9ea2-b2966bd82b55 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:55.740851 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 585/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 11:59:55 2022] GET /identity/v3/role_assignments?user.id=1cddaabdc2cb4c6782cbf8b59262dbf8&scope.project.id=6051a43122894939a366a52dd54b6f87&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 11:59:57.656820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f797c9-3910-4ef5-978a-e94326aab2c7 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:57.657375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f797c9-3910-4ef5-978a-e94326aab2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:57.657375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f797c9-3910-4ef5-978a-e94326aab2c7 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:57.658721 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 577/1162] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 11:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 11:59:57.663722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2c948-53a1-4b4d-80ef-80c991b8a5c5 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:57.664164 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2c948-53a1-4b4d-80ef-80c991b8a5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:57.664257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2c948-53a1-4b4d-80ef-80c991b8a5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:57.682168 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-fbb2c948-53a1-4b4d-80ef-80c991b8a5c5 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 11:59:57.683347 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-fbb2c948-53a1-4b4d-80ef-80c991b8a5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 11:59:57.749736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 586/1163] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 11:59:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 11:59:58.019050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f797c9-3910-4ef5-978a-e94326aab2c7 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:58.046079 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:58.124516 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f797c9-3910-4ef5-978a-e94326aab2c7 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:58.158132 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:58.158736 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:58.159011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:58.159284 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:58.228456 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:58.229948 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:58.252913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:58.327527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 578/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 11:59:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 11:59:58.333592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2c948-53a1-4b4d-80ef-80c991b8a5c5 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:58.417493 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:58.432312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2c948-53a1-4b4d-80ef-80c991b8a5c5 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:58.465299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c64c8-815e-4d03-860f-2a44f8e8d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:58.466475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c64c8-815e-4d03-860f-2a44f8e8d5dd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:58.466755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c64c8-815e-4d03-860f-2a44f8e8d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:58.467024 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c64c8-815e-4d03-860f-2a44f8e8d5dd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:58.473764 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c64c8-815e-4d03-860f-2a44f8e8d5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:58.474984 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c64c8-815e-4d03-860f-2a44f8e8d5dd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-280c64c8-815e-4d03-860f-2a44f8e8d5dd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 11:59:58.487125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 11:59:58.489169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 11:59:58.491097 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 587/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 11:59:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 11:59:58.497423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 11:59:58.594638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020b256-e4ca-4295-a451-0482c023db2c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 11:59:58.616768 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 11:59:58.628143 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df64ee-e210-4233-b512-c00a5e3b9ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 11:59:58.633883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df64ee-e210-4233-b512-c00a5e3b9ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 11:59:58.633883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df64ee-e210-4233-b512-c00a5e3b9ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 11:59:58.633883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df64ee-e210-4233-b512-c00a5e3b9ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 11:59:58.633883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df64ee-e210-4233-b512-c00a5e3b9ec2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 11:59:58.633883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df64ee-e210-4233-b512-c00a5e3b9ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 11:59:58.633883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df64ee-e210-4233-b512-c00a5e3b9ec2 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 11:59:58.642033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 579/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Sep 1 11:59:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:00.539985 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c8615-9b2e-498d-9753-232bddd31879 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:00.540510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c8615-9b2e-498d-9753-232bddd31879 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:00.540626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c8615-9b2e-498d-9753-232bddd31879 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:00.541915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 588/1167] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:00:00.547582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d17c2-6613-4d70-ae7b-65ea5866c6f8 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:00.547984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d17c2-6613-4d70-ae7b-65ea5866c6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:00.548128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d17c2-6613-4d70-ae7b-65ea5866c6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:00.566089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3a2d17c2-6613-4d70-ae7b-65ea5866c6f8 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:00:00.567272 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3a2d17c2-6613-4d70-ae7b-65ea5866c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:00:00.634663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 580/1168] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:00:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:00:00.899501 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c8615-9b2e-498d-9753-232bddd31879 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:01.000561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c8615-9b2e-498d-9753-232bddd31879 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:01.034488 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7295b5a-fab1-42e9-831e-79f3d83669c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:01.035086 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7295b5a-fab1-42e9-831e-79f3d83669c5 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:01.035370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7295b5a-fab1-42e9-831e-79f3d83669c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:01.035643 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7295b5a-fab1-42e9-831e-79f3d83669c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:01.036309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7295b5a-fab1-42e9-831e-79f3d83669c5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:01.037695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7295b5a-fab1-42e9-831e-79f3d83669c5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:01.098208 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 589/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Thu Sep 1 12:00:00 2022] POST /identity/v3/domains => generated 267 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 12:00:03.035424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826b6a-2403-4286-ba3b-499cf5759f83 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:03.036000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826b6a-2403-4286-ba3b-499cf5759f83 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:03.036000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826b6a-2403-4286-ba3b-499cf5759f83 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:03.037317 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 581/1170] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:00:03.042494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c82c4e-7d24-4800-8682-6176ec5e4150 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:03.042894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c82c4e-7d24-4800-8682-6176ec5e4150 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:03.043043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c82c4e-7d24-4800-8682-6176ec5e4150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:03.057338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:03.062755 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-b1c82c4e-7d24-4800-8682-6176ec5e4150 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:00:03.063947 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-b1c82c4e-7d24-4800-8682-6176ec5e4150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:00:03.130515 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 590/1171] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:00:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:00:03.396663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69826b6a-2403-4286-ba3b-499cf5759f83 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:03.496305 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69826b6a-2403-4286-ba3b-499cf5759f83 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:03.513389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:03.530378 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3122d9-a1ae-4a5d-b2a9-317815517d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:03.538487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3122d9-a1ae-4a5d-b2a9-317815517d1c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:03.538487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3122d9-a1ae-4a5d-b2a9-317815517d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:03.538487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3122d9-a1ae-4a5d-b2a9-317815517d1c None admin] PATH_INFO: `/v3/domains/ed4a4574fdbe4952b6f33d9e3d7a3f15` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:03.541624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3122d9-a1ae-4a5d-b2a9-317815517d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=ed4a4574fdbe4952b6f33d9e3d7a3f15)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:03.543071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3122d9-a1ae-4a5d-b2a9-317815517d1c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:03.544718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 582/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Sep 1 12:00:03 2022] GET /identity/v3/domains/ed4a4574fdbe4952b6f33d9e3d7a3f15 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:03.550619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c82c4e-7d24-4800-8682-6176ec5e4150 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:03.646567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c82c4e-7d24-4800-8682-6176ec5e4150 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:03.679180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35f696-6c97-4b34-a4e0-662645c1c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:03.679715 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35f696-6c97-4b34-a4e0-662645c1c5f3 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:03.679993 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35f696-6c97-4b34-a4e0-662645c1c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:03.680263 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35f696-6c97-4b34-a4e0-662645c1c5f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:03.680922 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35f696-6c97-4b34-a4e0-662645c1c5f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:03.682366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35f696-6c97-4b34-a4e0-662645c1c5f3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:03.688249 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:03.802144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 591/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 12:00:03 2022] POST /identity/v3/projects => generated 361 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 12:00:05.727640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375f974-7e9d-4098-961e-20066ad26066 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:05.728160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375f974-7e9d-4098-961e-20066ad26066 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:05.728262 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375f974-7e9d-4098-961e-20066ad26066 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:05.729536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 583/1174] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:00:05.735411 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5c64-336a-48e4-a6f5-789d3db078e3 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:05.735802 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5c64-336a-48e4-a6f5-789d3db078e3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:05.735963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5c64-336a-48e4-a6f5-789d3db078e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:05.753596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-73af5c64-336a-48e4-a6f5-789d3db078e3 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:00:05.754803 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-73af5c64-336a-48e4-a6f5-789d3db078e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:00:05.819814 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 592/1175] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:00:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:00:06.089887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375f974-7e9d-4098-961e-20066ad26066 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:06.188509 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375f974-7e9d-4098-961e-20066ad26066 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:06.221845 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87139bdd-b04a-4fe3-9ce1-9c8f9f464832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:06.222470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87139bdd-b04a-4fe3-9ce1-9c8f9f464832 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:06.222745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87139bdd-b04a-4fe3-9ce1-9c8f9f464832 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:06.223018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87139bdd-b04a-4fe3-9ce1-9c8f9f464832 None admin] PATH_INFO: `/v3/domains/ed4a4574fdbe4952b6f33d9e3d7a3f15` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:06.224110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87139bdd-b04a-4fe3-9ce1-9c8f9f464832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ed4a4574fdbe4952b6f33d9e3d7a3f15)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:06.225536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87139bdd-b04a-4fe3-9ce1-9c8f9f464832 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:06.227118 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 584/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Sep 1 12:00:06 2022] GET /identity/v3/domains/ed4a4574fdbe4952b6f33d9e3d7a3f15 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:06.233669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5c64-336a-48e4-a6f5-789d3db078e3 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:06.331117 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5c64-336a-48e4-a6f5-789d3db078e3 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:06.348611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:06.364493 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfcbd5-afa7-4b1d-885b-a266fb1b83e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:06.365173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfcbd5-afa7-4b1d-885b-a266fb1b83e6 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:06.365376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfcbd5-afa7-4b1d-885b-a266fb1b83e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:06.365670 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfcbd5-afa7-4b1d-885b-a266fb1b83e6 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:06.366330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfcbd5-afa7-4b1d-885b-a266fb1b83e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:06.367748 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfcbd5-afa7-4b1d-885b-a266fb1b83e6 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:06.434766 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 593/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 12:00:06 2022] POST /identity/v3/users => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 12:00:08.383585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51bf80-2c9b-4322-ae0c-fabdcb4ef888 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:08.384138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51bf80-2c9b-4322-ae0c-fabdcb4ef888 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:08.384138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51bf80-2c9b-4322-ae0c-fabdcb4ef888 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:08.385452 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 585/1178] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:00:08.391106 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6c0e9-6263-4bdd-9cb3-4ac4635c7dd7 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:08.391548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6c0e9-6263-4bdd-9cb3-4ac4635c7dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:08.391656 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6c0e9-6263-4bdd-9cb3-4ac4635c7dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:08.410072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-9aa6c0e9-6263-4bdd-9cb3-4ac4635c7dd7 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:00:08.411267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9aa6c0e9-6263-4bdd-9cb3-4ac4635c7dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:00:08.477150 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 594/1179] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:00:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:00:08.740627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51bf80-2c9b-4322-ae0c-fabdcb4ef888 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:08.787251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:08.841726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51bf80-2c9b-4322-ae0c-fabdcb4ef888 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:08.874069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:08.874694 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:08.874968 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:08.875236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:08.885581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:00:08.890324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:00:08.891462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:00:08.892364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 586/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 12:00:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:00:08.897750 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6c0e9-6263-4bdd-9cb3-4ac4635c7dd7 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:08.993650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6c0e9-6263-4bdd-9cb3-4ac4635c7dd7 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:08.995824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:09.026465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:09.027109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:09.027625 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:09.028562 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:09.029589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:00:09.029987 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:09.031390 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:09.039531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 595/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 12:00:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 12:00:09.046009 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:09.142535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85653fb-6d10-42c3-804e-610612146648 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:09.144813 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:09.175531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:09.176159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:09.176440 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:09.176711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] PATH_INFO: `/v3/users/e6ec6a111748455ea58d714aaee1c418` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:09.189628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] RBAC: Authorizing `identity:get_user(user_id=e6ec6a111748455ea58d714aaee1c418)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:09.191074 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:09.192898 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 587/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 12:00:09 2022] GET /identity/v3/users/e6ec6a111748455ea58d714aaee1c418 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:09.198962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:09.297425 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e156ed3-9bb5-4f7f-97fb-75bc61297a2b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:09.334929 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:09.335539 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:09.335818 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:09.336095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:09.368941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:09.402923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:09.404921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:09.503579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 596/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 12:00:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 12:00:09.509502 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:09.604821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eb809-dfa1-4cfc-8cd0-6bc003049843 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:09.637282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4c65e-b764-4330-abac-d4244ebc5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:09.638463 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4c65e-b764-4330-abac-d4244ebc5767 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:09.638740 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4c65e-b764-4330-abac-d4244ebc5767 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:09.639010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4c65e-b764-4330-abac-d4244ebc5767 None admin] PATH_INFO: `/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:09.640096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4c65e-b764-4330-abac-d4244ebc5767 None admin] RBAC: Authorizing `identity:get_project(project_id=171daf27b2d44c2e9dbd96d0a8ce825b)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:09.641513 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4c65e-b764-4330-abac-d4244ebc5767 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:09.643140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 588/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 12:00:09 2022] GET /identity/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:09.649193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:09.712288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:09.745341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b90f-7839-42dc-b65c-e3c233ffe245 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:09.778983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9cef0-c13f-42c4-80a8-5cdbc66eff8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:09.779585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9cef0-c13f-42c4-80a8-5cdbc66eff8c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:09.779868 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9cef0-c13f-42c4-80a8-5cdbc66eff8c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:09.780143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9cef0-c13f-42c4-80a8-5cdbc66eff8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:09.780862 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9cef0-c13f-42c4-80a8-5cdbc66eff8c None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=171daf27b2d44c2e9dbd96d0a8ce825b, user.id=e6ec6a111748455ea58d714aaee1c418) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:00:09.781172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9cef0-c13f-42c4-80a8-5cdbc66eff8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:09.782617 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9cef0-c13f-42c4-80a8-5cdbc66eff8c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:09.787921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 597/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 12:00:09 2022] GET /identity/v3/role_assignments?user.id=e6ec6a111748455ea58d714aaee1c418&scope.project.id=171daf27b2d44c2e9dbd96d0a8ce825b&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:11.658591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37c16-c450-4f79-9462-7a2d674dd461 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:11.659135 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37c16-c450-4f79-9462-7a2d674dd461 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:11.659135 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37c16-c450-4f79-9462-7a2d674dd461 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:11.660467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 589/1186] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:00:11.665888 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd9f1-deea-4538-9be9-d20623b8bfd2 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:11.666288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd9f1-deea-4538-9be9-d20623b8bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:11.666452 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd9f1-deea-4538-9be9-d20623b8bfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:11.685258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-ec2fd9f1-deea-4538-9be9-d20623b8bfd2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:00:11.686504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-ec2fd9f1-deea-4538-9be9-d20623b8bfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:00:11.752810 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 598/1187] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:00:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:00:12.020565 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37c16-c450-4f79-9462-7a2d674dd461 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:12.051038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:12.125984 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37c16-c450-4f79-9462-7a2d674dd461 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:12.160520 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:12.161154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:12.161433 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:12.161807 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:12.172698 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:00:12.177616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:00:12.178787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:00:12.180004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 590/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 12:00:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:00:12.184898 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd9f1-deea-4538-9be9-d20623b8bfd2 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:12.242266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:12.282505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd9f1-deea-4538-9be9-d20623b8bfd2 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:12.317054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:12.317681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:12.317967 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:12.318491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:12.319148 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:00:12.319454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:12.320805 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:12.329205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 599/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 12:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:12.335018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:12.424637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:12.433932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a833407-f2c8-42d2-bfb8-5807aec58883 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:12.470377 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:12.471013 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:12.471294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:12.471566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] PATH_INFO: `/v3/users/e6ec6a111748455ea58d714aaee1c418` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:12.472691 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] RBAC: Authorizing `identity:get_user(user_id=e6ec6a111748455ea58d714aaee1c418)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:12.474171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:12.475773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 591/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 12:00:12 2022] GET /identity/v3/users/e6ec6a111748455ea58d714aaee1c418 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:12.483537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:12.582962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83855e55-0215-4388-af26-0274e3f0d72f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:12.605155 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:12.616931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fcb31c-273c-4379-86cc-7824bcbd6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:12.617557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcb31c-273c-4379-86cc-7824bcbd6f50 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:12.617870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcb31c-273c-4379-86cc-7824bcbd6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:12.618142 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcb31c-273c-4379-86cc-7824bcbd6f50 None admin] PATH_INFO: `/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:12.619256 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fcb31c-273c-4379-86cc-7824bcbd6f50 None admin] RBAC: Authorizing `identity:get_project(project_id=171daf27b2d44c2e9dbd96d0a8ce825b)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:12.620684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fcb31c-273c-4379-86cc-7824bcbd6f50 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:12.622352 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 600/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 12:00:12 2022] GET /identity/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:12.628536 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:12.740546 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be56f6-f05f-4bbf-96b5-ce2cc1d55a34 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:12.773717 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c416ef3-d652-415c-8998-f1d8b561f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:12.774365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416ef3-d652-415c-8998-f1d8b561f43c None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:12.774638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416ef3-d652-415c-8998-f1d8b561f43c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:12.774907 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416ef3-d652-415c-8998-f1d8b561f43c None admin] PATH_INFO: `/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b/users/e6ec6a111748455ea58d714aaee1c418/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:12.776761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c416ef3-d652-415c-8998-f1d8b561f43c None admin] RBAC: Authorizing `identity:create_grant(project_id=171daf27b2d44c2e9dbd96d0a8ce825b, user_id=e6ec6a111748455ea58d714aaee1c418, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:12.778732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c416ef3-d652-415c-8998-f1d8b561f43c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:12.857719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 592/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 12:00:12 2022] PUT /identity/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b/users/e6ec6a111748455ea58d714aaee1c418/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 12:00:14.748322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d489ea-8e6c-4925-a816-eaef85ff09e0 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:14.748828 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d489ea-8e6c-4925-a816-eaef85ff09e0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:14.748920 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d489ea-8e6c-4925-a816-eaef85ff09e0 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:14.750241 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 601/1193] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:00:14.755436 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d2f93-31bb-402e-abca-b8e5ebf9eb2f None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:14.755850 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d2f93-31bb-402e-abca-b8e5ebf9eb2f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:14.755986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d2f93-31bb-402e-abca-b8e5ebf9eb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:14.773884 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d28d2f93-31bb-402e-abca-b8e5ebf9eb2f None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:00:14.775058 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d28d2f93-31bb-402e-abca-b8e5ebf9eb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:00:14.788676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:14.842537 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 593/1194] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:00:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:00:15.111818 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d489ea-8e6c-4925-a816-eaef85ff09e0 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:15.211153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d489ea-8e6c-4925-a816-eaef85ff09e0 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:15.244186 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:15.244813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:15.245094 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:15.245370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:15.255478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:00:15.260266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:00:15.261459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:00:15.263193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 602/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 12:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:00:15.267974 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d2f93-31bb-402e-abca-b8e5ebf9eb2f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:15.368121 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d2f93-31bb-402e-abca-b8e5ebf9eb2f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:15.401898 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:15.402508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:15.402786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:15.403581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:15.404240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:00:15.404543 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:15.405932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:15.413970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 594/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 12:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:15.420115 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:15.474195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:15.523520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d62820-05ca-4b4a-8d31-38e34c3e5aa8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:15.556713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:15.557339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:15.557645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:15.557925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] PATH_INFO: `/v3/users/e6ec6a111748455ea58d714aaee1c418` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:15.559045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] RBAC: Authorizing `identity:get_user(user_id=e6ec6a111748455ea58d714aaee1c418)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:15.560957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:15.562571 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 603/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Sep 1 12:00:15 2022] GET /identity/v3/users/e6ec6a111748455ea58d714aaee1c418 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:15.576038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:15.589650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:15.678018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f62f8-0f8a-4278-b38e-8744a48314ee None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:15.711997 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:15.712606 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:15.712881 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:15.713153 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:15.781771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:15.783222 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:15.795066 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:15.883062 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 595/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 12:00:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 12:00:15.893073 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:15.991230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe12a9e-4892-4a2f-816d-944ee7855e8b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:16.007864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:16.024459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cf2cf-3fe2-4aa5-8ad6-7e79c3012618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:16.025087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf2cf-3fe2-4aa5-8ad6-7e79c3012618 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:16.025369 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf2cf-3fe2-4aa5-8ad6-7e79c3012618 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:16.025666 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf2cf-3fe2-4aa5-8ad6-7e79c3012618 None admin] PATH_INFO: `/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:16.026780 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cf2cf-3fe2-4aa5-8ad6-7e79c3012618 None admin] RBAC: Authorizing `identity:get_project(project_id=171daf27b2d44c2e9dbd96d0a8ce825b)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:16.028743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cf2cf-3fe2-4aa5-8ad6-7e79c3012618 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:16.030396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 604/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Sep 1 12:00:15 2022] GET /identity/v3/projects/171daf27b2d44c2e9dbd96d0a8ce825b => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:16.036676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:00:16.137792 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de606a83-a81a-4889-bd86-1e13d5cd004d None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:00:16.148407 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:16.172020 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e153da-8bbe-4352-94c5-9c3fe7394726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:00:16.172619 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e153da-8bbe-4352-94c5-9c3fe7394726 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:00:16.172897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e153da-8bbe-4352-94c5-9c3fe7394726 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:00:16.173170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e153da-8bbe-4352-94c5-9c3fe7394726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:00:16.173921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e153da-8bbe-4352-94c5-9c3fe7394726 None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=171daf27b2d44c2e9dbd96d0a8ce825b, user.id=e6ec6a111748455ea58d714aaee1c418) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:00:16.174228 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e153da-8bbe-4352-94c5-9c3fe7394726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:00:16.175658 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e153da-8bbe-4352-94c5-9c3fe7394726 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:00:16.181810 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 596/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 12:00:16 2022] GET /identity/v3/role_assignments?user.id=e6ec6a111748455ea58d714aaee1c418&scope.project.id=171daf27b2d44c2e9dbd96d0a8ce825b&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:00:26.038904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=7 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:00:26.182642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=13 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:36.299054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e33e1-bbc5-4ed4-9109-907f2c0f6837 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:36.299652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e33e1-bbc5-4ed4-9109-907f2c0f6837 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:36.299652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e33e1-bbc5-4ed4-9109-907f2c0f6837 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:36.300926 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 605/1201] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:01:36.306020 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c13bc9-076a-4717-a1df-1577335a22cd None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:36.306423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c13bc9-076a-4717-a1df-1577335a22cd None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:36.306573 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c13bc9-076a-4717-a1df-1577335a22cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:36.324898 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-a5c13bc9-076a-4717-a1df-1577335a22cd None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:01:36.326115 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a5c13bc9-076a-4717-a1df-1577335a22cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:01:36.393838 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 597/1202] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:01:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 12:01:36.663965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e33e1-bbc5-4ed4-9109-907f2c0f6837 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:36.773701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e33e1-bbc5-4ed4-9109-907f2c0f6837 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:36.807617 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:36.808232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:36.808519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:36.808831 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:36.819425 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:36.820849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:36.836491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:36.837907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:36.839216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:36.839216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:36.839216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:36.839216 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 606/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:36.843625 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c13bc9-076a-4717-a1df-1577335a22cd None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:36.941802 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c13bc9-076a-4717-a1df-1577335a22cd None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:36.972325 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:36.982493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771e6d0-1b1b-4ee2-bd97-b22c100520ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:36.982493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771e6d0-1b1b-4ee2-bd97-b22c100520ea None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:36.982493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771e6d0-1b1b-4ee2-bd97-b22c100520ea None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:36.982493 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771e6d0-1b1b-4ee2-bd97-b22c100520ea None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:36.982833 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771e6d0-1b1b-4ee2-bd97-b22c100520ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:36.982833 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771e6d0-1b1b-4ee2-bd97-b22c100520ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:36.982833 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771e6d0-1b1b-4ee2-bd97-b22c100520ea None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:36.992372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 598/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:36.998237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:37.051041 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:37.096368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c66c30-87f9-4242-9272-17d0715aa58c None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:37.128767 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123648a3-858c-4e50-b97a-3faa911011a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:37.129364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123648a3-858c-4e50-b97a-3faa911011a5 None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:37.129665 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123648a3-858c-4e50-b97a-3faa911011a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:37.129943 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123648a3-858c-4e50-b97a-3faa911011a5 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:37.130621 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123648a3-858c-4e50-b97a-3faa911011a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:37.132038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123648a3-858c-4e50-b97a-3faa911011a5 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:37.258875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 607/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Sep 1 12:01:36 2022] POST /identity/v3/users => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 12:01:39.218273 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebda7f8-c677-431c-a335-838b91c8e918 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:39.218832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebda7f8-c677-431c-a335-838b91c8e918 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:39.218832 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebda7f8-c677-431c-a335-838b91c8e918 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:39.220201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 599/1206] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:01:39.225325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9768c7-d49e-4862-b0fb-a99cec4c3ca1 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:39.225738 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9768c7-d49e-4862-b0fb-a99cec4c3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:39.225910 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9768c7-d49e-4862-b0fb-a99cec4c3ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:39.244199 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-0e9768c7-d49e-4862-b0fb-a99cec4c3ca1 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:01:39.245386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0e9768c7-d49e-4862-b0fb-a99cec4c3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:01:39.314348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 608/1207] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:01:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:01:39.591647 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebda7f8-c677-431c-a335-838b91c8e918 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:39.693553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebda7f8-c677-431c-a335-838b91c8e918 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:39.726588 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:39.727225 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:39.732358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:39.732358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:39.739616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:01:39.744483 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:39.745714 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:39.746626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 600/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 12:01:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:39.751614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9768c7-d49e-4862-b0fb-a99cec4c3ca1 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:39.789521 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:39.848396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9768c7-d49e-4862-b0fb-a99cec4c3ca1 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:39.882017 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:39.882618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:39.882895 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:39.883163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:39.883811 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:39.884144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:39.885489 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:39.895884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 609/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 12:01:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:39.901641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:39.955231 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:40.008538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae561e9-59d2-4a40-83fa-d644c2b98f8c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:40.045146 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:40.049576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:40.049576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:40.049576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:40.058896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:40.060391 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:40.076749 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:40.081913 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:40.083089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:40.083089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:40.083089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:40.083089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 601/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:40.084931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:40.187568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:40.190548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a28a0-7acc-4918-b6d4-14db16490ce8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:40.225033 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:40.225706 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:40.226284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:40.226284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:40.226949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:40.227285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:40.228674 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:40.241217 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 610/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:40.248937 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:40.352169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ae888-06ee-4a15-a211-6a54dbf20fab None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:40.353794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:40.385616 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:40.386244 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:40.386520 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:40.386789 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:40.394828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:40.396291 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 12:01:40.407412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:40.408626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:40.409955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 602/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:40.421495 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:40.514632 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5247d-0115-471a-b9a4-b22603782e2d None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:40.547427 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:40.548022 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:40.548299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:40.548569 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:40.549301 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:40.550679 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:40.550997 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:40.569036 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 611/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Sep 1 12:01:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:40.574758 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:40.674976 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eded3-5053-446f-ad5c-acf273d67b14 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:40.714068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:40.714283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:40.714446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:40.714719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:40.746349 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:40.786018 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:40.787464 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:40.884933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 603/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 12:01:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 12:01:40.890983 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:40.909743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:40.989434 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6dbbe-9713-4fd1-853f-0589142da608 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:41.023894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:41.024543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:41.024823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:41.025103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:41.034982 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:41.036449 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:41.051963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:41.053105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:41.054084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:41.054084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:41.054084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:41.054084 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 612/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:41.059083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:41.118507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:41.155499 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e424de-0aa3-467a-ac19-cf0ed0393a2e None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:41.188345 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:41.188941 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:41.189217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:41.189545 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:41.190303 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:41.190608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:41.192238 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:41.203425 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 604/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:41.217736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:41.277330 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:41.309951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3b34f-fae3-4e6e-8d9b-dddcbb384f51 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:41.349186 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:41.349887 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:41.350171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:41.350446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:41.358072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:41.359584 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 12:01:41.372700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:41.373953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:41.375012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 613/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 12:01:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:41.380260 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:41.490684 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61991d8-4506-46d2-9c3b-e8e5befc6318 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:41.513690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:41.526626 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f86be2-721e-417e-8e83-bc3127014cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:41.527398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f86be2-721e-417e-8e83-bc3127014cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:41.527612 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f86be2-721e-417e-8e83-bc3127014cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:41.527885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f86be2-721e-417e-8e83-bc3127014cf0 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:41.528674 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f86be2-721e-417e-8e83-bc3127014cf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:41.528981 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f86be2-721e-417e-8e83-bc3127014cf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:41.530447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f86be2-721e-417e-8e83-bc3127014cf0 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:41.543303 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 605/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 12:01:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:41.549941 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:41.649695 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd5fb5-9371-4d7f-8e0e-b2d747efe8ed None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:41.682191 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:41.688289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829efc7-66ad-4997-8700-12a29a491bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:41.688911 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829efc7-66ad-4997-8700-12a29a491bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:41.689194 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829efc7-66ad-4997-8700-12a29a491bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:41.689471 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829efc7-66ad-4997-8700-12a29a491bc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:41.690241 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829efc7-66ad-4997-8700-12a29a491bc7 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=72ca589ac0054417b433852ec38b17a0) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:41.690556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829efc7-66ad-4997-8700-12a29a491bc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:41.692354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829efc7-66ad-4997-8700-12a29a491bc7 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:41.698957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 614/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 12:01:41 2022] GET /identity/v3/role_assignments?user.id=72ca589ac0054417b433852ec38b17a0&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:43.592046 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460720ec-7fe1-4979-8b07-3d7667bcf560 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:43.592610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460720ec-7fe1-4979-8b07-3d7667bcf560 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:43.592610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460720ec-7fe1-4979-8b07-3d7667bcf560 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:43.594045 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 606/1220] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:01:43.602272 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc9b05-f8d8-4753-a0c9-0f5c47050b41 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:43.602742 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc9b05-f8d8-4753-a0c9-0f5c47050b41 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:43.602857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc9b05-f8d8-4753-a0c9-0f5c47050b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:43.621329 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-6adc9b05-f8d8-4753-a0c9-0f5c47050b41 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:01:43.622555 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-6adc9b05-f8d8-4753-a0c9-0f5c47050b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:01:43.690789 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 615/1221] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:01:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:01:43.961995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460720ec-7fe1-4979-8b07-3d7667bcf560 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:44.063610 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460720ec-7fe1-4979-8b07-3d7667bcf560 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:44.099165 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:44.103949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:44.103949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:44.103949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:44.110735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:01:44.116154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:44.117294 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:44.118385 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 607/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 12:01:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:44.123376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc9b05-f8d8-4753-a0c9-0f5c47050b41 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:44.219836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc9b05-f8d8-4753-a0c9-0f5c47050b41 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:44.245728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:44.257168 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:44.257840 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:44.258122 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:44.258391 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:44.259069 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:44.259372 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:44.260773 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:44.270371 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 616/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 12:01:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:44.276671 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:44.325275 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:44.384814 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8f0c5-d058-4e55-8c66-a65d235d8a7c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:44.418481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:44.419096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:44.419375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:44.419685 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:44.433952 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:44.435408 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:44.450401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:44.451600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:44.452551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:44.452551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:44.452551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:44.452551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 608/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:44.457838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:44.558956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31931b-31d9-4c3e-aefa-dfdd92fedda8 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:44.593475 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:44.594114 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:44.594396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:44.594670 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:44.595315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:44.595621 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:44.597595 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:44.609332 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 617/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:44.615566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:44.713302 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:44.719988 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee79e8c-e799-45ae-a32d-8692d1c38df8 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:44.753413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:44.754083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:44.754363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:44.754635 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:44.762065 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:44.763506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 12:01:44.774986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:44.776087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:44.777069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 609/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:44.782174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:44.832364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:44.883613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bb5f1-4815-4469-89d8-c2309b35f3fe None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:44.917505 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:44.918143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:44.918425 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:44.918699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:44.919451 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:44.919757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:44.921175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:44.938504 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 618/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Sep 1 12:01:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:44.945089 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:45.050369 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c8bf-1d0d-403c-a5da-d8e7c9dd9806 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:45.084230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:45.084791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:45.085068 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:45.085341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:45.095691 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:45.097198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:45.111245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:45.116342 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:45.117579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:45.117579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:45.117579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:45.117579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 610/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:45.118662 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:45.218699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e9d6c-36f3-4fce-b367-a6023625f76e None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:45.221118 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:45.254252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:45.254871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:45.255154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:45.255448 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:45.256098 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:45.256404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:45.257837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:45.270119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 619/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 12:01:45.278569 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:45.285690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:45.380650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1324c91-3c77-4cbc-b579-756061af9328 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:45.413929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:45.414559 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:45.414870 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:45.415272 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:45.423038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:45.425084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 12:01:45.436539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:45.437644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:45.438741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 611/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 12:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:45.444677 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:45.550135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c168475-9677-4ec5-91e6-450e10245f8b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:45.584172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eaa046-b1e3-4166-bfa9-010b0c2b3141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:45.585185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa046-b1e3-4166-bfa9-010b0c2b3141 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:45.585185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa046-b1e3-4166-bfa9-010b0c2b3141 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:45.585472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa046-b1e3-4166-bfa9-010b0c2b3141 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:45.586025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eaa046-b1e3-4166-bfa9-010b0c2b3141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:45.586338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eaa046-b1e3-4166-bfa9-010b0c2b3141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:45.587815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eaa046-b1e3-4166-bfa9-010b0c2b3141 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:45.599732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 620/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 12:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:45.605849 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:45.675041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:45.709181 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f710-6ed4-4636-9fe5-9f43c629b28c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:45.744970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cfc59-0bfe-4ca2-aa99-d09c5a23b906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:45.745692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cfc59-0bfe-4ca2-aa99-d09c5a23b906 None admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:45.745973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cfc59-0bfe-4ca2-aa99-d09c5a23b906 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:45.746245 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cfc59-0bfe-4ca2-aa99-d09c5a23b906 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/72ca589ac0054417b433852ec38b17a0/roles/bc60629774714a078250dbc704217d2a` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:45.760534 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cfc59-0bfe-4ca2-aa99-d09c5a23b906 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=72ca589ac0054417b433852ec38b17a0, role_id=bc60629774714a078250dbc704217d2a)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:45.762229 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cfc59-0bfe-4ca2-aa99-d09c5a23b906 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:45.855004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 612/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 12:01:45 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/72ca589ac0054417b433852ec38b17a0/roles/bc60629774714a078250dbc704217d2a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 12:01:47.743352 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325c2f7-5f2b-4f7e-8f2c-3b0dbd27f780 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:47.743954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325c2f7-5f2b-4f7e-8f2c-3b0dbd27f780 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:47.743954 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325c2f7-5f2b-4f7e-8f2c-3b0dbd27f780 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:47.745227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 621/1233] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:01:47.750700 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4985f-a960-4f9b-aca8-f4cb507e4c8a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:47.751210 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4985f-a960-4f9b-aca8-f4cb507e4c8a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:47.751351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4985f-a960-4f9b-aca8-f4cb507e4c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:47.770449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-92f4985f-a960-4f9b-aca8-f4cb507e4c8a None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:01:47.771644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-92f4985f-a960-4f9b-aca8-f4cb507e4c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:01:47.840527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 613/1234] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:01:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:01:48.113339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325c2f7-5f2b-4f7e-8f2c-3b0dbd27f780 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:48.141709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:48.217733 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325c2f7-5f2b-4f7e-8f2c-3b0dbd27f780 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:48.254452 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:48.255112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:48.255397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:48.255668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:48.266083 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:01:48.271618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:48.272908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:48.274030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 622/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Sep 1 12:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:48.279488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4985f-a960-4f9b-aca8-f4cb507e4c8a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:48.327762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:48.382745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4985f-a960-4f9b-aca8-f4cb507e4c8a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:48.416991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:48.417636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:48.425773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:48.425773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:48.425773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:48.425773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:48.425773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:48.429363 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 614/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Sep 1 12:01:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:48.435264 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:48.526658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:48.543030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d21501-1d6f-4b68-a38e-35222da4f994 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:48.578258 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:48.578890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:48.579173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:48.579443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:48.589837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:48.591285 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:48.606240 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:48.612893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:48.614160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:48.614160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:48.614160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:48.614160 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 623/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:48.614648 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:48.723142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6136-9fce-4b86-8cd7-3ee78a444c5a None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:48.759572 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:48.760282 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:48.760460 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:48.760746 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:48.761383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:48.761787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:48.767005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:48.776497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 615/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:48.782580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:48.849641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:48.898163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cae16c-a80f-43ea-bb45-c5bd36e77abb None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:48.932404 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:48.933037 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:48.933317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:48.933592 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:48.936271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:48.941328 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:48.943054 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 12:01:48.954052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:48.963478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:48.964609 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 624/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:48.965255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:49.064424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5883-ae5d-4d6f-adf6-4e11cea93abb None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:49.100522 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:49.101124 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:49.101398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:49.101692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:49.102458 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:49.102761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:49.104175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:49.121260 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 616/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Sep 1 12:01:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:49.127136 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:49.228156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281535-ed59-4c5b-9324-fe49a6b10c51 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:49.260930 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:49.261529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:49.261841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:49.262116 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:49.322161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:49.330461 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:49.333089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:49.429131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 625/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 12:01:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 12:01:49.435136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:49.465004 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:49.535997 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214d83-b4d3-45eb-8346-6b8107986b12 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:49.569820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:49.570445 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:49.570720 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:49.570989 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:49.581456 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:49.582939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:49.599801 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:49.601057 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:49.602226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:49.602226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:49.602226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:49.602226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 617/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:49.607176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:49.704343 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:49.710590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1ca1a-934a-4c13-a149-b37061f22b55 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:49.747938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:49.748556 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:49.748846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:49.749121 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:49.749867 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:49.750172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:49.751530 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:49.763351 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 626/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:49.769901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:49.851420 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:49.872961 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f01bf9-2c42-4203-beb8-2f9d31a3c178 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:49.907635 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:49.908290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:49.908566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:49.908835 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:49.915315 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:49.916746 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 12:01:49.928900 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:49.930114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:49.931071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 618/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 12:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:49.935992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:50.040962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d5807-7321-417c-8029-e929277e9ddf None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:50.075621 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29b74c-10ab-41aa-8ff6-741f09dd20dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:50.076226 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b74c-10ab-41aa-8ff6-741f09dd20dd None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:50.076508 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b74c-10ab-41aa-8ff6-741f09dd20dd None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:50.076781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b74c-10ab-41aa-8ff6-741f09dd20dd None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:50.077474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29b74c-10ab-41aa-8ff6-741f09dd20dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:50.077809 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29b74c-10ab-41aa-8ff6-741f09dd20dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:50.079221 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29b74c-10ab-41aa-8ff6-741f09dd20dd None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:50.091021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 627/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 12:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:50.097234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:50.207164 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e6bab-1f4f-4013-810e-a0204d443457 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:50.240712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0df1d-eac7-494b-bf21-d8bb4016c9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:50.241319 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0df1d-eac7-494b-bf21-d8bb4016c9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:50.241592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0df1d-eac7-494b-bf21-d8bb4016c9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:50.241890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0df1d-eac7-494b-bf21-d8bb4016c9a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:50.242598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0df1d-eac7-494b-bf21-d8bb4016c9a6 None admin] RBAC: Adding query filter params (role.id=bc60629774714a078250dbc704217d2a, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=72ca589ac0054417b433852ec38b17a0) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:50.242901 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0df1d-eac7-494b-bf21-d8bb4016c9a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:50.244338 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0df1d-eac7-494b-bf21-d8bb4016c9a6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:50.250169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 619/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 12:01:50 2022] GET /identity/v3/role_assignments?user.id=72ca589ac0054417b433852ec38b17a0&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=bc60629774714a078250dbc704217d2a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:52.248490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ee3a8-dd6a-4414-a70c-13c2478064b1 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:52.249567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ee3a8-dd6a-4414-a70c-13c2478064b1 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:52.249567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ee3a8-dd6a-4414-a70c-13c2478064b1 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:52.250422 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 628/1247] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:01:52.255727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c041-3dea-49d0-968e-72f04aa9c8e6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:52.256241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c041-3dea-49d0-968e-72f04aa9c8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:52.256335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c041-3dea-49d0-968e-72f04aa9c8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:52.262730 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:52.275805 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3279c041-3dea-49d0-968e-72f04aa9c8e6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:01:52.276994 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3279c041-3dea-49d0-968e-72f04aa9c8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:01:52.347710 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 620/1248] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:01:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:01:52.613672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ee3a8-dd6a-4414-a70c-13c2478064b1 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:52.632454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:52.716986 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ee3a8-dd6a-4414-a70c-13c2478064b1 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:52.751670 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:52.752872 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:52.753152 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:52.753420 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:52.763529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:01:52.768913 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:52.770087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:52.770996 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 629/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 12:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 12:01:52.776220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c041-3dea-49d0-968e-72f04aa9c8e6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:52.880864 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c041-3dea-49d0-968e-72f04aa9c8e6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:52.930730 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:52.938039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:52.939024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:52.940040 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:52.940556 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:52.942095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:52.942591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:52.944992 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:52.959368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 621/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 12:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:52.965257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:53.052454 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:53.075668 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47e490-3c7b-4ffc-ad20-cdda3a1bf777 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:53.110223 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:53.110841 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:53.111660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:53.111905 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:53.121753 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:53.123210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:53.138813 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:53.140006 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:53.141082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:53.141082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:53.141082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:53.141082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 630/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:53.146147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:53.247296 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71edb-9d50-47f7-8f69-afbf7e70610c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:53.281299 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:53.281971 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:53.282236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:53.282466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:53.283587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:53.283888 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:53.285272 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:53.300708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 622/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:53.306823 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:53.409787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922fb45-b8cb-4988-87fa-a9e26fc4756b None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:53.435456 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:53.443225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:53.443859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:53.444140 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:53.444413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:53.452576 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:53.456042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 12:01:53.465270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:53.466326 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:53.467379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 631/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:53.472885 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:53.530637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:53.574676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dc6a5-e8f3-40e2-a49c-04d0f0005d4c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:53.608161 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:53.608755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:53.609028 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:53.609295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:53.611117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:53.611117 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:53.611909 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:53.630183 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 623/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Sep 1 12:01:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:53.636300 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:53.735774 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53863bbc-d96c-4023-bd00-361871a781ce None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:53.770394 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:53.771025 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:53.771312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:53.771589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:53.825557 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:53.837787 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:53.839227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:53.936291 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 632/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 12:01:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 12:01:53.942960 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:54.045310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2baeaf7-ab85-479f-87a7-25eeea0b3766 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:54.058289 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:54.079526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:54.080743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:54.081024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:54.081297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:54.090997 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:54.092447 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:54.106539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:54.107727 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:54.108713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:54.108713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:54.108713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:54.108713 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 624/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:54.113563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:54.200577 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:54.212377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb83b5-c257-45c6-92da-2944545b49d7 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:54.245205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:54.245918 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:54.246316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:54.246594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:54.247236 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:54.247542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:54.248893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:54.260949 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 633/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:54.266946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:54.369403 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde2f5-d447-42cf-acfd-b7ff80600d36 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:54.404000 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:54.404634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:54.404962 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:54.405264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:54.412003 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:54.413446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 12:01:54.425251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:54.426366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:01:54.427438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 625/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 12:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 12:01:54.432636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:54.534331 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d058d-80b5-4bbb-bdcb-d48321dafe37 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:54.568503 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334d6d6-8d5e-4391-86b5-0230d6d39640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:54.569103 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334d6d6-8d5e-4391-86b5-0230d6d39640 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:54.569381 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334d6d6-8d5e-4391-86b5-0230d6d39640 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:54.569687 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334d6d6-8d5e-4391-86b5-0230d6d39640 None admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:54.570377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334d6d6-8d5e-4391-86b5-0230d6d39640 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:54.570680 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334d6d6-8d5e-4391-86b5-0230d6d39640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:54.572100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334d6d6-8d5e-4391-86b5-0230d6d39640 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:54.584266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 634/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 12:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:54.590867 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:54.616627 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:54.702914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce687dee-8fd4-4963-afd0-5c00c7a6a23c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:54.737567 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43013a5e-85d3-4340-b78d-2bb6ae278a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:54.738229 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43013a5e-85d3-4340-b78d-2bb6ae278a8d None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:54.738506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43013a5e-85d3-4340-b78d-2bb6ae278a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:54.738776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43013a5e-85d3-4340-b78d-2bb6ae278a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:54.739487 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43013a5e-85d3-4340-b78d-2bb6ae278a8d None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=72ca589ac0054417b433852ec38b17a0) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:54.739790 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43013a5e-85d3-4340-b78d-2bb6ae278a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:54.741389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43013a5e-85d3-4340-b78d-2bb6ae278a8d None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:54.746783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 626/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 12:01:54 2022] GET /identity/v3/role_assignments?user.id=72ca589ac0054417b433852ec38b17a0&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:56.622139 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7652a8-b1fc-42b7-83de-c2d0a50f2af5 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:56.622669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7652a8-b1fc-42b7-83de-c2d0a50f2af5 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:56.622669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7652a8-b1fc-42b7-83de-c2d0a50f2af5 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:56.624012 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 635/1261] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:01:56.629177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706480cc-4655-4e35-ac67-922dac093a00 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:56.629577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706480cc-4655-4e35-ac67-922dac093a00 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:56.629757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706480cc-4655-4e35-ac67-922dac093a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:56.649116 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-706480cc-4655-4e35-ac67-922dac093a00 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:01:56.650353 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-706480cc-4655-4e35-ac67-922dac093a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:01:56.733766 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 627/1262] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:01:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:01:56.999125 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7652a8-b1fc-42b7-83de-c2d0a50f2af5 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:57.011014 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:57.103030 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7652a8-b1fc-42b7-83de-c2d0a50f2af5 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:57.138630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:57.139261 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:57.139538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:57.139812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:57.150629 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:01:57.155835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:57.158029 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:57.158938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 636/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 12:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:57.164177 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706480cc-4655-4e35-ac67-922dac093a00 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:57.182390 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:57.266093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706480cc-4655-4e35-ac67-922dac093a00 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:57.303708 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:57.304313 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:57.304591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:57.304865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:57.305527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:57.305864 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:57.307236 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:57.316649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 628/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 12:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:57.324021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:57.399864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:57.435052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b79dd1-7bb5-4f36-b41c-2a63cd903bec None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:57.475159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:57.475953 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:57.476269 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:57.476542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:57.490213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:57.491657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:57.507482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:57.508699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:57.510063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:57.510063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:57.510063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:57.510063 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 637/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:57.514635 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:57.600337 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:57.620084 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d9979-bcc9-4f3f-a648-6590d87e3736 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:57.653660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:57.654307 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:57.654553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:57.654828 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:57.655473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:57.655776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:57.657187 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:57.670251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 629/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:57.676549 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:57.780320 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b3b26-5495-439b-8a3f-77b9fd803768 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:57.800743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:57.815818 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:57.816708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:57.816987 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:57.817257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:57.825374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:57.826884 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 12:01:57.838028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:57.839209 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:57.840396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 638/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:57.846260 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:57.944642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57665b7c-f2ef-4bd6-b534-70941c517226 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:57.978247 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:57.978860 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:57.979137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:57.979408 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] PATH_INFO: `/v3/users` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:57.980128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:57.980433 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:57.981892 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:57.991239 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:57.999887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 630/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Sep 1 12:01:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:58.005805 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:58.108724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8b318-5575-4e81-a7c2-7c72858bc1c3 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:58.141958 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:58.143108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:58.143390 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:58.143663 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:58.154149 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:58.155706 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:01:58.170924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:58.172109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:01:58.173384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:01:58.173384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:01:58.173384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:58.173384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 639/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 12:01:58.178263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:58.276397 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e5acf-7609-4c87-b0eb-cbeee5d72000 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:58.310554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:58.311158 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:58.311435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:58.311702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:58.312344 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:58.312644 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:58.314069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:58.325851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 631/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:58.332340 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:58.365213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:58.433229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c397402-8a7d-4a2d-965f-a7dae790a096 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:58.469977 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:58.470657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:58.470942 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:58.471220 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:58.477790 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:58.479213 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 12:01:58.490462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:01:58.491523 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:01:58.492672 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 640/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 12:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:01:58.497752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:58.557674 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:58.599496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda164be-1f4f-4de5-8ca2-495b905007c9 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:58.632426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1129ce1-56c7-443a-9f17-d640b36ebd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:58.633029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129ce1-56c7-443a-9f17-d640b36ebd67 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:58.633309 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129ce1-56c7-443a-9f17-d640b36ebd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:58.633582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1129ce1-56c7-443a-9f17-d640b36ebd67 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:58.634296 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129ce1-56c7-443a-9f17-d640b36ebd67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:01:58.634602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129ce1-56c7-443a-9f17-d640b36ebd67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:58.636033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1129ce1-56c7-443a-9f17-d640b36ebd67 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:58.648600 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 632/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 12:01:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:01:58.654871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:01:58.749796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:01:58.759011 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973317-71da-4935-b9d4-90dbcda9544f None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:01:58.793131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3115d-781d-48a2-a5dc-784eaa766710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:01:58.794413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3115d-781d-48a2-a5dc-784eaa766710 None admin] REQUEST_METHOD: `PUT` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:01:58.794413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3115d-781d-48a2-a5dc-784eaa766710 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:01:58.794413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3115d-781d-48a2-a5dc-784eaa766710 None admin] PATH_INFO: `/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/72ca589ac0054417b433852ec38b17a0/roles/1ef322c2b68c44eda56fcb1825b667c6` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:01:58.796341 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3115d-781d-48a2-a5dc-784eaa766710 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f9de6c1f1c645c095ec56f3ff7451da, user_id=72ca589ac0054417b433852ec38b17a0, role_id=1ef322c2b68c44eda56fcb1825b667c6)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:01:58.797827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3115d-781d-48a2-a5dc-784eaa766710 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:01:58.894052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 641/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Sep 1 12:01:58 2022] PUT /identity/v3/projects/5f9de6c1f1c645c095ec56f3ff7451da/users/72ca589ac0054417b433852ec38b17a0/roles/1ef322c2b68c44eda56fcb1825b667c6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 12:02:00.830577 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d4943-8d2e-4808-bde0-4708cd45a4c6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:00.831457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d4943-8d2e-4808-bde0-4708cd45a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:00.831457 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d4943-8d2e-4808-bde0-4708cd45a4c6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:00.832471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 633/1274] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:02:00.838846 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542ea53-007a-4784-947c-a126b0ba6998 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:00.839334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542ea53-007a-4784-947c-a126b0ba6998 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:00.839837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542ea53-007a-4784-947c-a126b0ba6998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:00.869921 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-4542ea53-007a-4784-947c-a126b0ba6998 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:02:00.871991 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-4542ea53-007a-4784-947c-a126b0ba6998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:02:00.942257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 642/1275] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:02:01.208614 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d4943-8d2e-4808-bde0-4708cd45a4c6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:01.316061 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d4943-8d2e-4808-bde0-4708cd45a4c6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:01.337531 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:01.355920 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:01.356560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:01.356839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:01.357112 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:01.367527 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 12:02:01.372314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:01.373293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:01.374234 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 634/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Sep 1 12:02:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:01.380691 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542ea53-007a-4784-947c-a126b0ba6998 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:01.481074 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:01.499442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542ea53-007a-4784-947c-a126b0ba6998 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:01.533826 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:01.534439 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:01.534721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:01.534992 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:01.535652 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:01.535957 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:01.537313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:01.545797 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 643/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Sep 1 12:02:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:01.551674 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:01.656029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49dd5-d20e-42c5-9ea7-9365b54403da None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:01.689970 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:01.690641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:01.690918 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:01.691433 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:01.702078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:01.703525 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:02:01.718139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:01.719217 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:02:01.720159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:02:01.720159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:02:01.720159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:01.720159 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 635/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:01.725374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:01.832594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782ac41-1194-471a-a06e-0f76c4498f21 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:01.868559 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:01.869275 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:01.869664 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:01.869825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:01.870494 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:01.870804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:01.872190 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:01.885827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 644/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:01.891712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:01.924359 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:01.992533 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619e769-9f07-49ee-bfef-0715af1d03a6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:02.025561 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:02.026219 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:02.026496 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:02.026765 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:02.034178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:02.035865 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 12:02:02.047935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:02.049682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:02.050640 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 636/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:02:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:02.055979 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:02.059917 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:02.154829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2062e-350f-41e9-8246-0386ec7763c1 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:02.189702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:02.190309 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:02.190587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:02.190859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:02.191839 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:02.192144 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:02.193551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:02.210817 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 645/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Sep 1 12:02:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:02.216764 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:02.309088 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:02.319472 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cbd83-ac03-43f7-b4f3-d1b1792bf0a6 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:02.353230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:02.358726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:02.358726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:02.358726 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:02.425631 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:02.427052 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:02.516762 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:02.528143 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 637/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Sep 1 12:02:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 12:02:02.534883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:02.594896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:02.633873 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e809f-926c-4405-aaed-43e00a3e5a84 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:02.668366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:02.668999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:02.669281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:02.669558 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:02.679614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:02.681028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 12:02:02.696701 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:02.697944 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 12:02:02.699237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 12:02:02.699237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 12:02:02.699237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:02:02.699237 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 646/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:02.703698 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:02.806831 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbab6d-0e20-407f-9e04-738fd870deb7 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:02.841951 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:02.842573 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:02.842909 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:02.843242 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] PATH_INFO: `/v3/domains` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:02.843915 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:02.844216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:02.846237 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:02.859576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 638/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Sep 1 12:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:02.865931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:02.967376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151a506-cda7-4dd4-b577-173630a20f86 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:02.980883 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:03.000252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:03.000934 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:03.004334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:03.004334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:03.008529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:03.009999 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 12:02:03.021746 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:03.022885 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: ERROR keystone.server.flask.application Sep 01 12:02:03.024134 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 647/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Sep 1 12:02:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:03.029054 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:03.081924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:03.131333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8650-d81a-4be1-93f0-1050d32bd7ad None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:03.164389 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f83c36-baa3-43f9-a8d6-4d2e267ea240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:03.164890 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f83c36-baa3-43f9-a8d6-4d2e267ea240 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:03.165171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f83c36-baa3-43f9-a8d6-4d2e267ea240 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:03.165442 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f83c36-baa3-43f9-a8d6-4d2e267ea240 None admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:03.166162 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f83c36-baa3-43f9-a8d6-4d2e267ea240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:03.166466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f83c36-baa3-43f9-a8d6-4d2e267ea240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:03.167886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f83c36-baa3-43f9-a8d6-4d2e267ea240 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:03.181205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 639/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Sep 1 12:02:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:03.187483 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:03.296386 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49325516-e6fa-42b6-85c9-3ab28a475600 None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:03.336781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3536151-a33b-4dfe-a85e-8ddcf6fc123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:03.340544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536151-a33b-4dfe-a85e-8ddcf6fc123c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:03.340544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536151-a33b-4dfe-a85e-8ddcf6fc123c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:03.340544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3536151-a33b-4dfe-a85e-8ddcf6fc123c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:03.340544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536151-a33b-4dfe-a85e-8ddcf6fc123c None admin] RBAC: Adding query filter params (role.id=1ef322c2b68c44eda56fcb1825b667c6, scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da, user.id=72ca589ac0054417b433852ec38b17a0) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:03.340544 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536151-a33b-4dfe-a85e-8ddcf6fc123c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:03.341016 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3536151-a33b-4dfe-a85e-8ddcf6fc123c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:03.348938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 648/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Sep 1 12:02:03 2022] GET /identity/v3/role_assignments?user.id=72ca589ac0054417b433852ec38b17a0&scope.project.id=5f9de6c1f1c645c095ec56f3ff7451da&role.id=1ef322c2b68c44eda56fcb1825b667c6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:05.297725 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc5bd7-8bcb-4f50-9b95-2eaa4ba32e19 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:05.298931 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc5bd7-8bcb-4f50-9b95-2eaa4ba32e19 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:05.298931 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc5bd7-8bcb-4f50-9b95-2eaa4ba32e19 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:05.300147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 640/1288] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:02:05.305831 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d280096-0b90-45c2-a750-8b0fc9f6579e None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:05.306302 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d280096-0b90-45c2-a750-8b0fc9f6579e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:05.306410 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d280096-0b90-45c2-a750-8b0fc9f6579e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:05.325188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-8d280096-0b90-45c2-a750-8b0fc9f6579e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:02:05.326447 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-8d280096-0b90-45c2-a750-8b0fc9f6579e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:02:05.360590 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:05.400791 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 649/1289] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:02:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:02:05.668723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc5bd7-8bcb-4f50-9b95-2eaa4ba32e19 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:05.772642 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc5bd7-8bcb-4f50-9b95-2eaa4ba32e19 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:05.784383 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:05.808358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:05.808981 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:05.809255 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:05.809524 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:05.810137 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:05.811502 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:05.820817 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:05.822096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:05.823063 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 641/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 12:02:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:05.828118 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d280096-0b90-45c2-a750-8b0fc9f6579e None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:05.928021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d280096-0b90-45c2-a750-8b0fc9f6579e None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:05.967427 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f289c-0675-4d59-91d2-b57e2de2056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:05.968066 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f289c-0675-4d59-91d2-b57e2de2056a None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:05.968349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f289c-0675-4d59-91d2-b57e2de2056a None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:05.968624 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f289c-0675-4d59-91d2-b57e2de2056a None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:05.969426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f289c-0675-4d59-91d2-b57e2de2056a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:05.969770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f289c-0675-4d59-91d2-b57e2de2056a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:05.971186 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f289c-0675-4d59-91d2-b57e2de2056a None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:05.976863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 650/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Sep 1 12:02:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 12:02:05.982939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:06.094783 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2057dd-581e-40ef-accd-720aaa7bef9c None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:06.129240 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405396e5-28b8-4db8-8de5-2f7f7db4a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:06.129869 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405396e5-28b8-4db8-8de5-2f7f7db4a1ef None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:06.130147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405396e5-28b8-4db8-8de5-2f7f7db4a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:06.130415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405396e5-28b8-4db8-8de5-2f7f7db4a1ef None admin] PATH_INFO: `/v3/services` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:06.131037 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405396e5-28b8-4db8-8de5-2f7f7db4a1ef None admin] RBAC: Adding query filter params (type=placement) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:06.131336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405396e5-28b8-4db8-8de5-2f7f7db4a1ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:06.132693 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405396e5-28b8-4db8-8de5-2f7f7db4a1ef None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:06.137122 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 642/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Sep 1 12:02:05 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:08.035409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4996e-4c5f-4623-a190-7a543229a781 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:08.035938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4996e-4c5f-4623-a190-7a543229a781 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:08.035938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4996e-4c5f-4623-a190-7a543229a781 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:08.037281 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 651/1293] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:02:08.042693 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb81cf-5be5-4202-a1a6-35494e54d590 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:08.043171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb81cf-5be5-4202-a1a6-35494e54d590 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:08.043271 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb81cf-5be5-4202-a1a6-35494e54d590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:08.061719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-e9eb81cf-5be5-4202-a1a6-35494e54d590 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:02:08.062982 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e9eb81cf-5be5-4202-a1a6-35494e54d590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:02:08.093215 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:08.131150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 643/1294] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:02:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:02:08.402583 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4996e-4c5f-4623-a190-7a543229a781 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:08.448858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:08.522908 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4996e-4c5f-4623-a190-7a543229a781 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:08.556985 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c56313-e856-40ce-a91a-5539c7583c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:08.559105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c56313-e856-40ce-a91a-5539c7583c5d None admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:08.559105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c56313-e856-40ce-a91a-5539c7583c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:08.559105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c56313-e856-40ce-a91a-5539c7583c5d None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:08.559105 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c56313-e856-40ce-a91a-5539c7583c5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:08.560176 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c56313-e856-40ce-a91a-5539c7583c5d None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:08.805289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 652/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Sep 1 12:02:08 2022] POST /identity/v3/services => generated 246 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 12:02:10.810586 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca4807-017e-4e47-8c53-61b20b8074cb None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:10.811139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca4807-017e-4e47-8c53-61b20b8074cb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:10.811139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca4807-017e-4e47-8c53-61b20b8074cb None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:10.812548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 644/1296] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:02:10.818277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f7eb-8116-4afa-a9d5-fcb129811039 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:10.818721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f7eb-8116-4afa-a9d5-fcb129811039 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:10.818835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f7eb-8116-4afa-a9d5-fcb129811039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:10.837856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-0999f7eb-8116-4afa-a9d5-fcb129811039 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:02:10.838911 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0999f7eb-8116-4afa-a9d5-fcb129811039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:02:10.913143 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 653/1297] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:02:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:02:11.184278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca4807-017e-4e47-8c53-61b20b8074cb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:11.290863 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca4807-017e-4e47-8c53-61b20b8074cb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:11.326250 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:11.326892 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:11.327169 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:11.327445 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:11.328038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:11.330070 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:11.339515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:11.340659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:11.341755 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 645/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 12:02:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:11.347076 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999f7eb-8116-4afa-a9d5-fcb129811039 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:11.365804 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:11.446364 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999f7eb-8116-4afa-a9d5-fcb129811039 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:11.484393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:11.485405 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:11.492945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:11.492945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:11.492945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] RBAC: Adding query filter params (name=placement) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:11.492945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:11.492945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:11.500182 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 654/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Sep 1 12:02:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:02:11.506902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:11.539637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:11.606946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ac515-bbd7-429e-b8f3-1a78caa1e648 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:11.641028 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c1639-1dc4-4539-baf6-b77329ccf864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:11.641701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1639-1dc4-4539-baf6-b77329ccf864 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:11.641985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1639-1dc4-4539-baf6-b77329ccf864 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:11.642257 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1639-1dc4-4539-baf6-b77329ccf864 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:11.642950 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c1639-1dc4-4539-baf6-b77329ccf864 None admin] RBAC: Adding query filter params (interface=public, service_id=92e5836f8d814de3960453fa5c11beb5, region_id=RegionOne) {{(pid=81862) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:11.643254 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c1639-1dc4-4539-baf6-b77329ccf864 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:11.644622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c1639-1dc4-4539-baf6-b77329ccf864 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:11.649504 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 646/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Sep 1 12:02:11 2022] GET /identity/v3/endpoints?service_id=92e5836f8d814de3960453fa5c11beb5&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 12:02:11.657324 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:11.744495 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:11.762723 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98ac3c-8c2f-4d88-b037-a0d2cee053de None admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:11.795135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9d09a-8ff8-4963-b3c7-037f72538b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:11.795739 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d09a-8ff8-4963-b3c7-037f72538b56 None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:11.796021 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d09a-8ff8-4963-b3c7-037f72538b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:11.796295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d09a-8ff8-4963-b3c7-037f72538b56 None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:11.796896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9d09a-8ff8-4963-b3c7-037f72538b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:11.798976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9d09a-8ff8-4963-b3c7-037f72538b56 None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:11.805745 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 655/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Sep 1 12:02:11 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 12:02:13.725042 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f767d13-3550-434c-8a2c-886e1ee40dbf None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:13.725566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f767d13-3550-434c-8a2c-886e1ee40dbf None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:13.725566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f767d13-3550-434c-8a2c-886e1ee40dbf None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:13.727011 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 647/1302] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:02:13.732366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438946ed-089d-4992-a15c-ec04912af5df None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:13.732807 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438946ed-089d-4992-a15c-ec04912af5df None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:13.732963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438946ed-089d-4992-a15c-ec04912af5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:13.751702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-438946ed-089d-4992-a15c-ec04912af5df None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:02:13.752901 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-438946ed-089d-4992-a15c-ec04912af5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:02:13.823912 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 656/1303] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:02:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 12:02:14.092597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f767d13-3550-434c-8a2c-886e1ee40dbf None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:14.194408 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:14.202655 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f767d13-3550-434c-8a2c-886e1ee40dbf None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:14.238754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:14.239382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:14.239660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:14.239930 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:14.240508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:14.241907 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 12:02:14.250637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 12:02:14.256532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application with Lock( Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self._enter() Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application return self.creator() Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application created_value = creator( Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: ERROR keystone.server.flask.application Sep 01 12:02:14.257729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 648/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Sep 1 12:02:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 12:02:14.258777 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438946ed-089d-4992-a15c-ec04912af5df None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:14.334903 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:14.357112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438946ed-089d-4992-a15c-ec04912af5df None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:14.392290 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33be83-aefc-4e6a-a62b-e86e1c46332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:14.392964 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33be83-aefc-4e6a-a62b-e86e1c46332c None admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:14.393245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33be83-aefc-4e6a-a62b-e86e1c46332c None admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:14.393513 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33be83-aefc-4e6a-a62b-e86e1c46332c None admin] PATH_INFO: `/v3/services` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:14.394173 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33be83-aefc-4e6a-a62b-e86e1c46332c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:02:14.394474 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33be83-aefc-4e6a-a62b-e86e1c46332c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:14.395820 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33be83-aefc-4e6a-a62b-e86e1c46332c None admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:14.401017 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 657/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Sep 1 12:02:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 12:02:14.407190 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:02:14.511386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e56a2-425d-44f5-811f-a6b18dc221c5 None admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:02:14.546001 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb0e70-b9b2-4b4d-ac5e-30222304e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:02:14.546646 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb0e70-b9b2-4b4d-ac5e-30222304e33b None admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:02:14.546925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb0e70-b9b2-4b4d-ac5e-30222304e33b None admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:02:14.547199 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb0e70-b9b2-4b4d-ac5e-30222304e33b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:02:14.548397 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb0e70-b9b2-4b4d-ac5e-30222304e33b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:02:14.549794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb0e70-b9b2-4b4d-ac5e-30222304e33b None admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:02:14.693734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 649/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 12:02:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 12:02:24.416172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=30 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:02:24.669535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:31.956286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d1e1-a987-4e51-bfbb-aad601d816a2 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:31.957188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d1e1-a987-4e51-bfbb-aad601d816a2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:31.957188 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d1e1-a987-4e51-bfbb-aad601d816a2 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:31.960051 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 658/1307] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:31.968877 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9360b04-e652-4aeb-8028-13d2706f9523 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:31.970555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9360b04-e652-4aeb-8028-13d2706f9523 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:31.970555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9360b04-e652-4aeb-8028-13d2706f9523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:32.112566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-d9360b04-e652-4aeb-8028-13d2706f9523 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:32.114587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d9360b04-e652-4aeb-8028-13d2706f9523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:32.322614 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 650/1308] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:32.623437 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61237e-9565-4035-b31a-7501477c9bf9 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:32.623733 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61237e-9565-4035-b31a-7501477c9bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:32.624034 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61237e-9565-4035-b31a-7501477c9bf9 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:32.625583 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 659/1309] 10.222.0.48 () {54 vars in 965 bytes} [Thu Sep 1 12:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 12:10:32.632743 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6f78a-991f-4721-9d79-44f1c51ecb92 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:32.633411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6f78a-991f-4721-9d79-44f1c51ecb92 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:32.634103 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6f78a-991f-4721-9d79-44f1c51ecb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:32.775862 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-44a6f78a-991f-4721-9d79-44f1c51ecb92 None None] MFA Rules not processed for user `75e0ceaa2c6745dc89ade0c43a17572b`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:32.777921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-44a6f78a-991f-4721-9d79-44f1c51ecb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:32.817551 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:32.929166 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 651/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Sep 1 12:10:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:10:32.940195 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61237e-9565-4035-b31a-7501477c9bf9 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:32.943396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-4a61237e-9565-4035-b31a-7501477c9bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:33.146586 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61237e-9565-4035-b31a-7501477c9bf9 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:33.278430 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be698b-418c-4b75-9159-0c930128e161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:33.279650 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be698b-418c-4b75-9159-0c930128e161 service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:33.280265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be698b-418c-4b75-9159-0c930128e161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:33.280881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be698b-418c-4b75-9159-0c930128e161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:33.284362 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-24be698b-418c-4b75-9159-0c930128e161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:33.403370 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:33.509567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be698b-418c-4b75-9159-0c930128e161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:33.511058 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be698b-418c-4b75-9159-0c930128e161 service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:33.542568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 660/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:36.115491 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa436be-2051-4b3c-bcd0-d1e22cf3fc8e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:36.116029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa436be-2051-4b3c-bcd0-d1e22cf3fc8e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:36.116142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa436be-2051-4b3c-bcd0-d1e22cf3fc8e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:36.117793 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 652/1312] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:36.123754 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0caf51-0fee-41ee-adec-10adc6f47178 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:36.124413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0caf51-0fee-41ee-adec-10adc6f47178 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:36.124413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0caf51-0fee-41ee-adec-10adc6f47178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:36.220170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2c0caf51-0fee-41ee-adec-10adc6f47178 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:36.222310 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2c0caf51-0fee-41ee-adec-10adc6f47178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:36.379392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 661/1313] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:36.671549 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa436be-2051-4b3c-bcd0-d1e22cf3fc8e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:36.674473 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3fa436be-2051-4b3c-bcd0-d1e22cf3fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:36.855734 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa436be-2051-4b3c-bcd0-d1e22cf3fc8e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:36.975370 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:36.979737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97201-9675-4c8f-9330-b45c50dda7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:36.980663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97201-9675-4c8f-9330-b45c50dda7be service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:36.981072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97201-9675-4c8f-9330-b45c50dda7be service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:36.981475 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97201-9675-4c8f-9330-b45c50dda7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:36.984295 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-15f97201-9675-4c8f-9330-b45c50dda7be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:37.136232 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f97201-9675-4c8f-9330-b45c50dda7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:37.138193 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f97201-9675-4c8f-9330-b45c50dda7be service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:37.159168 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 653/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:39.754945 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2db0-87cd-4e3a-b6c5-1580e0c9c6fd None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:39.755519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2db0-87cd-4e3a-b6c5-1580e0c9c6fd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:39.755519 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2db0-87cd-4e3a-b6c5-1580e0c9c6fd None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:39.757073 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 662/1315] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:39.763787 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb7599-c7b6-4236-b89f-ccf71b82a618 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:39.764576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb7599-c7b6-4236-b89f-ccf71b82a618 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:39.764576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb7599-c7b6-4236-b89f-ccf71b82a618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:39.845729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-79cb7599-c7b6-4236-b89f-ccf71b82a618 None None] MFA Rules not processed for user `0c11045dcf324aa8bd8417b3e8b91dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:39.846990 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-79cb7599-c7b6-4236-b89f-ccf71b82a618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:39.903466 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:39.950878 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 654/1316] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:40.251567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4fb5-37d3-472b-a101-3b4576e91320 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:40.251849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4fb5-37d3-472b-a101-3b4576e91320 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:40.252129 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4fb5-37d3-472b-a101-3b4576e91320 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:40.253484 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 663/1317] 10.222.0.48 () {54 vars in 965 bytes} [Thu Sep 1 12:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:40.259636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8df6-15e1-4e9a-bd93-a7010870a480 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:40.259922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8df6-15e1-4e9a-bd93-a7010870a480 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:40.260373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8df6-15e1-4e9a-bd93-a7010870a480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:40.339737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-aa4f8df6-15e1-4e9a-bd93-a7010870a480 None None] MFA Rules not processed for user `75e0ceaa2c6745dc89ade0c43a17572b`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:40.340973 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-aa4f8df6-15e1-4e9a-bd93-a7010870a480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:40.419591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 655/1318] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Sep 1 12:10:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:40.428902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4fb5-37d3-472b-a101-3b4576e91320 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:40.430793 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d35e4fb5-37d3-472b-a101-3b4576e91320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:40.440636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:40.547374 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4fb5-37d3-472b-a101-3b4576e91320 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:40.621588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162a689-fce5-404b-a68b-f879b665194f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:40.622228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162a689-fce5-404b-a68b-f879b665194f service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:40.622510 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162a689-fce5-404b-a68b-f879b665194f service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:40.622789 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162a689-fce5-404b-a68b-f879b665194f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:40.624428 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0162a689-fce5-404b-a68b-f879b665194f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:40.690177 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:40.737192 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162a689-fce5-404b-a68b-f879b665194f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:40.738655 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162a689-fce5-404b-a68b-f879b665194f service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:40.757728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 664/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:43.836935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c96a3c-87ad-4c7c-8d9c-8254681e73b7 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:43.837470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c96a3c-87ad-4c7c-8d9c-8254681e73b7 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:43.837470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c96a3c-87ad-4c7c-8d9c-8254681e73b7 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:43.838846 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 656/1320] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:43.843952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3cd1c-5ab4-4f87-b9d5-602964ea2b7d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:43.844403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3cd1c-5ab4-4f87-b9d5-602964ea2b7d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:43.844403 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3cd1c-5ab4-4f87-b9d5-602964ea2b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:43.909397 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-dec3cd1c-5ab4-4f87-b9d5-602964ea2b7d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:43.910656 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-dec3cd1c-5ab4-4f87-b9d5-602964ea2b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:44.006228 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 665/1321] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:44.284894 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c96a3c-87ad-4c7c-8d9c-8254681e73b7 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:44.286467 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f6c96a3c-87ad-4c7c-8d9c-8254681e73b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:44.339220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:44.394195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c96a3c-87ad-4c7c-8d9c-8254681e73b7 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:44.470638 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfe227-f156-4a99-8625-d3ee5bcdfdb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:44.471251 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfe227-f156-4a99-8625-d3ee5bcdfdb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:44.471553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfe227-f156-4a99-8625-d3ee5bcdfdb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:44.471827 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfe227-f156-4a99-8625-d3ee5bcdfdb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:44.473530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-61bfe227-f156-4a99-8625-d3ee5bcdfdb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:44.573699 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:44.586748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfe227-f156-4a99-8625-d3ee5bcdfdb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:44.588198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfe227-f156-4a99-8625-d3ee5bcdfdb2 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:44.606175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 657/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:47.612948 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c18b8-d738-41bc-ad1e-b38f6dae7462 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:47.613480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c18b8-d738-41bc-ad1e-b38f6dae7462 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:47.613480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c18b8-d738-41bc-ad1e-b38f6dae7462 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:47.614880 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 666/1323] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:47.620416 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2c99c-f877-4504-9db4-a5da3fdbb540 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:47.620880 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2c99c-f877-4504-9db4-a5da3fdbb540 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:47.621010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2c99c-f877-4504-9db4-a5da3fdbb540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:47.689455 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-52c2c99c-f877-4504-9db4-a5da3fdbb540 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:47.691357 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-52c2c99c-f877-4504-9db4-a5da3fdbb540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:47.790202 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 658/1324] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:48.073915 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c18b8-d738-41bc-ad1e-b38f6dae7462 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:48.075622 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-415c18b8-d738-41bc-ad1e-b38f6dae7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:48.099231 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:48.191985 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c18b8-d738-41bc-ad1e-b38f6dae7462 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:48.266972 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0bd66-6f6c-45b2-a6cf-671f02f28c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:48.267951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0bd66-6f6c-45b2-a6cf-671f02f28c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:48.268242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0bd66-6f6c-45b2-a6cf-671f02f28c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:48.268520 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0bd66-6f6c-45b2-a6cf-671f02f28c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:48.270202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-c8e0bd66-6f6c-45b2-a6cf-671f02f28c56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:48.344492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:48.383559 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0bd66-6f6c-45b2-a6cf-671f02f28c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:48.384900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0bd66-6f6c-45b2-a6cf-671f02f28c56 service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:48.403743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 667/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:50.439540 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d5c19-3223-43f1-9b6e-ed0ca4eec231 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:50.440150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d5c19-3223-43f1-9b6e-ed0ca4eec231 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:50.440150 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d5c19-3223-43f1-9b6e-ed0ca4eec231 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:50.441463 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 659/1326] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:50.447162 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904acc8d-1530-4a12-b9ef-082c045c3942 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:50.447648 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904acc8d-1530-4a12-b9ef-082c045c3942 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:50.447729 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904acc8d-1530-4a12-b9ef-082c045c3942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:50.524934 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-904acc8d-1530-4a12-b9ef-082c045c3942 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:50.526108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-904acc8d-1530-4a12-b9ef-082c045c3942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:50.636548 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 668/1327] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:50.929512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d5c19-3223-43f1-9b6e-ed0ca4eec231 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:50.931542 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e39d5c19-3223-43f1-9b6e-ed0ca4eec231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:50.996293 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:51.044044 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d5c19-3223-43f1-9b6e-ed0ca4eec231 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:51.152622 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907b580-5a1c-4987-ac97-84009df7cbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:51.153285 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b580-5a1c-4987-ac97-84009df7cbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:51.153566 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b580-5a1c-4987-ac97-84009df7cbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:51.153871 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907b580-5a1c-4987-ac97-84009df7cbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:51.155579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c907b580-5a1c-4987-ac97-84009df7cbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:51.280593 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:51.283505 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907b580-5a1c-4987-ac97-84009df7cbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:51.284975 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907b580-5a1c-4987-ac97-84009df7cbf5 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:51.304418 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 660/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:53.320736 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c59c6c-d774-44da-b1d3-0e95041b2850 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:53.321267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c59c6c-d774-44da-b1d3-0e95041b2850 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:53.321267 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c59c6c-d774-44da-b1d3-0e95041b2850 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:53.322815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 669/1329] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 12:10:53.328460 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768f92-9508-4724-a712-0a9db77cf36a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:53.329560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768f92-9508-4724-a712-0a9db77cf36a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:53.329560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768f92-9508-4724-a712-0a9db77cf36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:53.405456 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-8c768f92-9508-4724-a712-0a9db77cf36a None None] MFA Rules not processed for user `0c11045dcf324aa8bd8417b3e8b91dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:53.406750 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8c768f92-9508-4724-a712-0a9db77cf36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:53.514266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 661/1330] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:53.800721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c59c6c-d774-44da-b1d3-0e95041b2850 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:53.802348 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-89c59c6c-d774-44da-b1d3-0e95041b2850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:53.846903 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:53.936312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c59c6c-d774-44da-b1d3-0e95041b2850 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:54.019150 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac267b-7f3c-45b8-8c9a-b80e3d5be7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:54.020037 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac267b-7f3c-45b8-8c9a-b80e3d5be7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:54.020315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac267b-7f3c-45b8-8c9a-b80e3d5be7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:54.020585 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac267b-7f3c-45b8-8c9a-b80e3d5be7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:54.022252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-01ac267b-7f3c-45b8-8c9a-b80e3d5be7c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:54.101315 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:54.135297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac267b-7f3c-45b8-8c9a-b80e3d5be7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:54.137170 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac267b-7f3c-45b8-8c9a-b80e3d5be7c6 service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:54.155912 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 670/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:56.574286 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e2c32-e4c7-4993-a31b-959b73d24372 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:56.574844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e2c32-e4c7-4993-a31b-959b73d24372 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:56.574844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e2c32-e4c7-4993-a31b-959b73d24372 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:56.576176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 662/1332] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:56.583132 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b403-3bd0-4e42-bba8-aff8e9263e3d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:56.583617 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b403-3bd0-4e42-bba8-aff8e9263e3d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:56.583711 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b403-3bd0-4e42-bba8-aff8e9263e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:56.651004 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-02d7b403-3bd0-4e42-bba8-aff8e9263e3d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:56.652224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-02d7b403-3bd0-4e42-bba8-aff8e9263e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:56.749681 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 671/1333] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:10:57.037160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e2c32-e4c7-4993-a31b-959b73d24372 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:57.038760 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-592e2c32-e4c7-4993-a31b-959b73d24372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:57.110751 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:57.149142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e2c32-e4c7-4993-a31b-959b73d24372 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:10:57.224814 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140630f-91e7-4739-ba15-cd2500da082e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:10:57.225426 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140630f-91e7-4739-ba15-cd2500da082e service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:57.225739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140630f-91e7-4739-ba15-cd2500da082e service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:57.226039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140630f-91e7-4739-ba15-cd2500da082e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:57.227678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a140630f-91e7-4739-ba15-cd2500da082e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:57.337825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140630f-91e7-4739-ba15-cd2500da082e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:10:57.339207 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140630f-91e7-4739-ba15-cd2500da082e service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:10:57.350156 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:57.358424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 663/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:10:59.376398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47300611-2f0e-4ee8-95a1-a7197e8ed689 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:59.377092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47300611-2f0e-4ee8-95a1-a7197e8ed689 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:59.377092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47300611-2f0e-4ee8-95a1-a7197e8ed689 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:59.378232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 672/1335] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:10:59.383925 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227286c-653b-430a-9aea-a947f7cecf02 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:10:59.384702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227286c-653b-430a-9aea-a947f7cecf02 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:10:59.384702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227286c-653b-430a-9aea-a947f7cecf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:10:59.456940 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2227286c-653b-430a-9aea-a947f7cecf02 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:10:59.458310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2227286c-653b-430a-9aea-a947f7cecf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:59.573219 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 664/1336] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:10:59.865229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47300611-2f0e-4ee8-95a1-a7197e8ed689 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:10:59.866919 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-47300611-2f0e-4ee8-95a1-a7197e8ed689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:10:59.918077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:10:59.987719 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47300611-2f0e-4ee8-95a1-a7197e8ed689 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:00.066580 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc7a2-c3b1-46fa-8f1e-72e59f3050c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:00.067626 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc7a2-c3b1-46fa-8f1e-72e59f3050c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:00.067909 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc7a2-c3b1-46fa-8f1e-72e59f3050c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:00.068180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc7a2-c3b1-46fa-8f1e-72e59f3050c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:00.069835 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-948dc7a2-c3b1-46fa-8f1e-72e59f3050c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:00.157052 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:00.183382 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dc7a2-c3b1-46fa-8f1e-72e59f3050c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:00.184812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dc7a2-c3b1-46fa-8f1e-72e59f3050c4 service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:00.203156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 673/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:02.812745 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7754f13-3599-4af8-b28a-bcfabe902291 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:02.813887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7754f13-3599-4af8-b28a-bcfabe902291 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:02.813991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7754f13-3599-4af8-b28a-bcfabe902291 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:02.815290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 665/1338] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:02.820647 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7692262-a5e8-4627-ac49-7ce71b389da3 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:02.821120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7692262-a5e8-4627-ac49-7ce71b389da3 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:02.821224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7692262-a5e8-4627-ac49-7ce71b389da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:02.887120 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-b7692262-a5e8-4627-ac49-7ce71b389da3 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:02.888335 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-b7692262-a5e8-4627-ac49-7ce71b389da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:02.987620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 674/1339] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:03.265898 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7754f13-3599-4af8-b28a-bcfabe902291 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:03.267441 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b7754f13-3599-4af8-b28a-bcfabe902291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:03.346029 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:03.374410 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7754f13-3599-4af8-b28a-bcfabe902291 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:03.450264 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083babe2-5c37-45bb-8423-c99a9fa0eb9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:03.450650 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083babe2-5c37-45bb-8423-c99a9fa0eb9b service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:03.450940 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083babe2-5c37-45bb-8423-c99a9fa0eb9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:03.451215 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083babe2-5c37-45bb-8423-c99a9fa0eb9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:03.452843 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-083babe2-5c37-45bb-8423-c99a9fa0eb9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:03.569767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083babe2-5c37-45bb-8423-c99a9fa0eb9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:03.571367 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083babe2-5c37-45bb-8423-c99a9fa0eb9b service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:03.591112 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 666/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:07.801827 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b887f37-92a8-48da-8387-92081ac70693 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:07.802393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b887f37-92a8-48da-8387-92081ac70693 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:07.802393 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b887f37-92a8-48da-8387-92081ac70693 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:07.803716 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 675/1341] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:07.809266 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb89b0-316e-42e9-ac16-43938ccf8442 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:07.809812 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb89b0-316e-42e9-ac16-43938ccf8442 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:07.809919 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb89b0-316e-42e9-ac16-43938ccf8442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:07.815823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:07.881062 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-dcdb89b0-316e-42e9-ac16-43938ccf8442 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:07.882407 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-dcdb89b0-316e-42e9-ac16-43938ccf8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:07.988539 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 667/1342] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:08.267724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b887f37-92a8-48da-8387-92081ac70693 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:08.269376 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2b887f37-92a8-48da-8387-92081ac70693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:08.320480 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:08.379433 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b887f37-92a8-48da-8387-92081ac70693 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:08.454978 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc46c6b-ecf3-4bd5-a8c4-9754d748c452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:08.455597 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46c6b-ecf3-4bd5-a8c4-9754d748c452 service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:08.455878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46c6b-ecf3-4bd5-a8c4-9754d748c452 service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:08.456153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46c6b-ecf3-4bd5-a8c4-9754d748c452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:08.457796 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3dc46c6b-ecf3-4bd5-a8c4-9754d748c452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:08.562517 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:08.582686 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc46c6b-ecf3-4bd5-a8c4-9754d748c452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:08.584128 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc46c6b-ecf3-4bd5-a8c4-9754d748c452 service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:08.605035 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 676/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:11.867072 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b4654-ab6b-45e5-9405-8804804c39d2 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:11.867639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b4654-ab6b-45e5-9405-8804804c39d2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:11.867639 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b4654-ab6b-45e5-9405-8804804c39d2 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:11.868951 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 668/1344] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:11.874325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868861cd-009c-4af6-b604-e0a90dd1c9c2 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:11.874800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868861cd-009c-4af6-b604-e0a90dd1c9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:11.874896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868861cd-009c-4af6-b604-e0a90dd1c9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:11.946930 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-868861cd-009c-4af6-b604-e0a90dd1c9c2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:11.948232 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-868861cd-009c-4af6-b604-e0a90dd1c9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:12.056709 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 677/1345] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:12.342748 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b4654-ab6b-45e5-9405-8804804c39d2 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:12.346058 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-226b4654-ab6b-45e5-9405-8804804c39d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:12.457429 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:12.477481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b4654-ab6b-45e5-9405-8804804c39d2 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:12.565443 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51486ed-e521-4547-a431-64a72d14941d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:12.565998 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51486ed-e521-4547-a431-64a72d14941d service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:12.566278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51486ed-e521-4547-a431-64a72d14941d service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:12.566553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51486ed-e521-4547-a431-64a72d14941d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:12.568209 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c51486ed-e521-4547-a431-64a72d14941d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:12.706192 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51486ed-e521-4547-a431-64a72d14941d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:12.709126 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51486ed-e521-4547-a431-64a72d14941d service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:12.743776 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 669/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:17.270727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6660b2-94cb-4e7d-9909-1a937b490363 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:17.271270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6660b2-94cb-4e7d-9909-1a937b490363 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:17.271270 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6660b2-94cb-4e7d-9909-1a937b490363 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:17.272669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 678/1347] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 12:11:17.279104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c90b-ebcf-411a-9434-0e5640d7c1ed None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:17.288897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c90b-ebcf-411a-9434-0e5640d7c1ed None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:17.288897 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c90b-ebcf-411a-9434-0e5640d7c1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:17.297005 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:17.359757 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-db18c90b-ebcf-411a-9434-0e5640d7c1ed None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:17.360929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-db18c90b-ebcf-411a-9434-0e5640d7c1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:17.479518 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 670/1348] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:17.768589 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6660b2-94cb-4e7d-9909-1a937b490363 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:17.770175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-aa6660b2-94cb-4e7d-9909-1a937b490363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:17.849836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:17.927527 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6660b2-94cb-4e7d-9909-1a937b490363 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:18.025516 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81b115-933e-4911-a412-bab48c273add service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:18.026224 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81b115-933e-4911-a412-bab48c273add service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:18.026614 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81b115-933e-4911-a412-bab48c273add service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:18.026779 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81b115-933e-4911-a412-bab48c273add service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:18.028462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-fe81b115-933e-4911-a412-bab48c273add service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:18.148202 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:18.151821 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81b115-933e-4911-a412-bab48c273add service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:18.153325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81b115-933e-4911-a412-bab48c273add service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:18.173045 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 679/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:23.284351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b2bfd-95dc-4c1b-8e07-6909576f5bac None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:23.284351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b2bfd-95dc-4c1b-8e07-6909576f5bac None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:23.285116 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b2bfd-95dc-4c1b-8e07-6909576f5bac None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:23.287034 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 671/1350] 10.222.0.48 () {54 vars in 921 bytes} [Thu Sep 1 12:11:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:23.294262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c082-193a-41aa-9874-dbfbc6d107a5 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:23.294262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c082-193a-41aa-9874-dbfbc6d107a5 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:23.294848 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c082-193a-41aa-9874-dbfbc6d107a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:23.403070 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-0144c082-193a-41aa-9874-dbfbc6d107a5 None None] MFA Rules not processed for user `72ca589ac0054417b433852ec38b17a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:23.404500 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0144c082-193a-41aa-9874-dbfbc6d107a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:23.549763 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 680/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Sep 1 12:11:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:27.477411 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=59 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:28.011659 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0ca12-0e2c-4fb9-8688-452c7e46f1ed None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:28.012290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0ca12-0e2c-4fb9-8688-452c7e46f1ed None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:28.012712 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0ca12-0e2c-4fb9-8688-452c7e46f1ed None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:28.014585 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 672/1352] 10.222.0.48 () {54 vars in 921 bytes} [Thu Sep 1 12:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:28.022178 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31dc35-ca63-40a2-a4dd-b4418ae5970e None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:28.022724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31dc35-ca63-40a2-a4dd-b4418ae5970e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:28.023342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31dc35-ca63-40a2-a4dd-b4418ae5970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:28.133630 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:28.181453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2b31dc35-ca63-40a2-a4dd-b4418ae5970e None None] MFA Rules not processed for user `72ca589ac0054417b433852ec38b17a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:28.182860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2b31dc35-ca63-40a2-a4dd-b4418ae5970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:28.328574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 681/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Sep 1 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:30.573883 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8749-a178-4901-b5f5-d466897a54e0 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:30.574630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8749-a178-4901-b5f5-d466897a54e0 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:30.574630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8749-a178-4901-b5f5-d466897a54e0 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:30.575786 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 673/1354] 10.222.0.48 () {54 vars in 921 bytes} [Thu Sep 1 12:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 12:11:30.582274 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87056bfd-a47e-4842-8a08-5fa6b455eb4e None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:30.583108 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87056bfd-a47e-4842-8a08-5fa6b455eb4e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:30.583271 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87056bfd-a47e-4842-8a08-5fa6b455eb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:30.673179 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-87056bfd-a47e-4842-8a08-5fa6b455eb4e None None] MFA Rules not processed for user `72ca589ac0054417b433852ec38b17a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:30.674479 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-87056bfd-a47e-4842-8a08-5fa6b455eb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:30.727225 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:30.807366 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 682/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Sep 1 12:11:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:34.459131 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa129b7a-cf0d-4a4f-821e-183d239e86d5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:34.459797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa129b7a-cf0d-4a4f-821e-183d239e86d5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:34.459991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa129b7a-cf0d-4a4f-821e-183d239e86d5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:34.462329 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 674/1356] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:34.470598 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd75c7-d715-49c5-9c55-e08d45181da9 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:34.471353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd75c7-d715-49c5-9c55-e08d45181da9 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:34.471353 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd75c7-d715-49c5-9c55-e08d45181da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:34.594215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-1ffd75c7-d715-49c5-9c55-e08d45181da9 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:34.596324 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-1ffd75c7-d715-49c5-9c55-e08d45181da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:34.759535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 683/1357] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:35.106077 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff26a6-559b-48e2-b451-6ca2e4dfe10f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:35.106364 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff26a6-559b-48e2-b451-6ca2e4dfe10f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:35.106792 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff26a6-559b-48e2-b451-6ca2e4dfe10f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:35.107997 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 675/1358] 10.222.0.48 () {54 vars in 962 bytes} [Thu Sep 1 12:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:35.117087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c394fa-f16e-4321-ba2a-34212af369a4 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:35.117438 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c394fa-f16e-4321-ba2a-34212af369a4 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:35.117693 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c394fa-f16e-4321-ba2a-34212af369a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:35.142599 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:35.212538 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-67c394fa-f16e-4321-ba2a-34212af369a4 None None] MFA Rules not processed for user `0c8225879a66415c8a1dc28584e4c626`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:35.213963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-67c394fa-f16e-4321-ba2a-34212af369a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:35.351882 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 684/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Sep 1 12:11:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:35.360562 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff26a6-559b-48e2-b451-6ca2e4dfe10f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:35.362222 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-08ff26a6-559b-48e2-b451-6ca2e4dfe10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:35.535369 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff26a6-559b-48e2-b451-6ca2e4dfe10f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:35.628555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:35.683607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55503cb8-5e32-4355-b640-04e9f7eaa966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:35.683607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503cb8-5e32-4355-b640-04e9f7eaa966 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:35.683607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503cb8-5e32-4355-b640-04e9f7eaa966 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:35.683607 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503cb8-5e32-4355-b640-04e9f7eaa966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:35.684468 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-55503cb8-5e32-4355-b640-04e9f7eaa966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:35.796124 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55503cb8-5e32-4355-b640-04e9f7eaa966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:35.798906 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55503cb8-5e32-4355-b640-04e9f7eaa966 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:35.821220 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 676/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:38.037363 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ebb8-6882-4760-a0b4-b5527e4926c0 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:38.038024 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ebb8-6882-4760-a0b4-b5527e4926c0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:38.038024 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ebb8-6882-4760-a0b4-b5527e4926c0 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:38.039517 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 685/1361] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:38.045138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efeb73-6e29-4d87-b05c-267c12fed9ad None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:38.045844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efeb73-6e29-4d87-b05c-267c12fed9ad None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:38.045844 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efeb73-6e29-4d87-b05c-267c12fed9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:38.121179 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-11efeb73-6e29-4d87-b05c-267c12fed9ad None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:38.122320 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-11efeb73-6e29-4d87-b05c-267c12fed9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:38.163834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:38.231636 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 677/1362] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:38.521441 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583ebb8-6882-4760-a0b4-b5527e4926c0 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:38.523782 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3583ebb8-6882-4760-a0b4-b5527e4926c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:38.599222 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:38.715561 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583ebb8-6882-4760-a0b4-b5527e4926c0 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:38.875858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6888faa-5bff-4847-a68b-764850457519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:38.876468 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888faa-5bff-4847-a68b-764850457519 service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:38.876747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888faa-5bff-4847-a68b-764850457519 service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:38.877019 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888faa-5bff-4847-a68b-764850457519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:38.878721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-b6888faa-5bff-4847-a68b-764850457519 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:38.904903 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:39.028362 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6888faa-5bff-4847-a68b-764850457519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:39.031721 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6888faa-5bff-4847-a68b-764850457519 service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:39.064325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 686/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:41.537259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445560e-73da-47f3-90d2-60da13b3fd67 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:41.538333 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445560e-73da-47f3-90d2-60da13b3fd67 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:41.538823 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445560e-73da-47f3-90d2-60da13b3fd67 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:41.540310 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 678/1364] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:41.549586 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9eee3-350a-4837-a150-4d273a211517 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:41.550463 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9eee3-350a-4837-a150-4d273a211517 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:41.550963 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9eee3-350a-4837-a150-4d273a211517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:41.635185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-0cf9eee3-350a-4837-a150-4d273a211517 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:41.636732 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0cf9eee3-350a-4837-a150-4d273a211517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:41.760380 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:41.769646 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 687/1365] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:42.161372 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fde0a-847d-48ca-9811-27cce5de1eef None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:42.162733 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fde0a-847d-48ca-9811-27cce5de1eef None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:42.163515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fde0a-847d-48ca-9811-27cce5de1eef None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:42.166128 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 679/1366] 10.222.0.48 () {54 vars in 962 bytes} [Thu Sep 1 12:11:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:42.173096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcdbea-cdd8-4aa7-8847-1b91e8d8eaa0 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:42.174875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcdbea-cdd8-4aa7-8847-1b91e8d8eaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:42.174875 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcdbea-cdd8-4aa7-8847-1b91e8d8eaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:42.317512 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-aebcdbea-cdd8-4aa7-8847-1b91e8d8eaa0 None None] MFA Rules not processed for user `0c8225879a66415c8a1dc28584e4c626`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:42.319849 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-aebcdbea-cdd8-4aa7-8847-1b91e8d8eaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:42.548654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 688/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Sep 1 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:11:42.565718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fde0a-847d-48ca-9811-27cce5de1eef None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:42.568926 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8e0fde0a-847d-48ca-9811-27cce5de1eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:42.800884 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fde0a-847d-48ca-9811-27cce5de1eef None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:42.827933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:42.953512 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d74fc4-699e-43dc-87e7-4af41fed8b39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:42.954557 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74fc4-699e-43dc-87e7-4af41fed8b39 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:42.955041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74fc4-699e-43dc-87e7-4af41fed8b39 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:42.955602 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74fc4-699e-43dc-87e7-4af41fed8b39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:42.958680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d0d74fc4-699e-43dc-87e7-4af41fed8b39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:43.078488 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d74fc4-699e-43dc-87e7-4af41fed8b39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:43.080197 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d74fc4-699e-43dc-87e7-4af41fed8b39 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:43.107322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 680/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:43.606210 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7757a3a-abe5-4b48-b88e-aae9e1b0c116 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:43.606890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7757a3a-abe5-4b48-b88e-aae9e1b0c116 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:43.606890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7757a3a-abe5-4b48-b88e-aae9e1b0c116 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:43.609537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 689/1369] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:43.618222 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db109a-1599-4368-9ddb-586767ade5a2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:43.618806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db109a-1599-4368-9ddb-586767ade5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:43.619188 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db109a-1599-4368-9ddb-586767ade5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:43.673316 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:43.742538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-27db109a-1599-4368-9ddb-586767ade5a2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:43.744114 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-27db109a-1599-4368-9ddb-586767ade5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:43.930705 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 681/1370] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:44.668050 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4734-d6fc-4b9b-ae93-b97f52a68ded None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:44.668587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4734-d6fc-4b9b-ae93-b97f52a68ded None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:44.668587 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed4734-d6fc-4b9b-ae93-b97f52a68ded None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:44.670185 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 690/1371] 10.222.0.48 () {54 vars in 964 bytes} [Thu Sep 1 12:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:44.676167 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd22adf-8278-42ec-8c0d-29d3a8cc726a None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:44.676682 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd22adf-8278-42ec-8c0d-29d3a8cc726a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:44.676784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd22adf-8278-42ec-8c0d-29d3a8cc726a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:44.782806 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ddd22adf-8278-42ec-8c0d-29d3a8cc726a None None] MFA Rules not processed for user `86e8ec32ae1f46e9a1a3ada4fc7201c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:44.784767 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ddd22adf-8278-42ec-8c0d-29d3a8cc726a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:44.841950 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:44.878986 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 682/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Sep 1 12:11:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:44.886405 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed4734-d6fc-4b9b-ae93-b97f52a68ded None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:44.887938 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-74ed4734-d6fc-4b9b-ae93-b97f52a68ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:44.956612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:45.019292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed4734-d6fc-4b9b-ae93-b97f52a68ded None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:45.098077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a560d9-0770-4bcc-a9a4-9471dc2e96c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86e8ec32ae1f46e9a1a3ada4fc7201c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:45.098959 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a560d9-0770-4bcc-a9a4-9471dc2e96c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:45.099322 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a560d9-0770-4bcc-a9a4-9471dc2e96c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:45.099698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a560d9-0770-4bcc-a9a4-9471dc2e96c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:45.101700 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d9a560d9-0770-4bcc-a9a4-9471dc2e96c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:45.228077 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:45.248203 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a560d9-0770-4bcc-a9a4-9471dc2e96c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:45.251698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a560d9-0770-4bcc-a9a4-9471dc2e96c6 service cinder] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:45.283183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 691/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 12:11:45.451672 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd6034-9de0-461c-8f3d-fc78991694a9 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:45.451956 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd6034-9de0-461c-8f3d-fc78991694a9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:45.452230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd6034-9de0-461c-8f3d-fc78991694a9 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:45.454188 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 683/1374] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:45.459649 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf015e-8f8c-46bb-b5e2-c6a1ffb9265d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:45.459940 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf015e-8f8c-46bb-b5e2-c6a1ffb9265d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:45.460220 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf015e-8f8c-46bb-b5e2-c6a1ffb9265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:45.533325 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-78cf015e-8f8c-46bb-b5e2-c6a1ffb9265d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:45.534611 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-78cf015e-8f8c-46bb-b5e2-c6a1ffb9265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:45.639043 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 692/1375] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:11:45.930222 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd6034-9de0-461c-8f3d-fc78991694a9 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:45.931752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2fdd6034-9de0-461c-8f3d-fc78991694a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:46.094545 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd6034-9de0-461c-8f3d-fc78991694a9 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:46.174180 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:46.260590 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abfbd6-bdd2-4784-8362-bb73bba4df73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:46.261336 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abfbd6-bdd2-4784-8362-bb73bba4df73 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:46.261753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abfbd6-bdd2-4784-8362-bb73bba4df73 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:46.262198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abfbd6-bdd2-4784-8362-bb73bba4df73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:46.264083 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-66abfbd6-bdd2-4784-8362-bb73bba4df73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:46.376519 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abfbd6-bdd2-4784-8362-bb73bba4df73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:46.378604 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abfbd6-bdd2-4784-8362-bb73bba4df73 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:46.400253 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 684/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:48.585396 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8b2f7-9f06-4a6f-b57f-f41068adfa81 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:48.586245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8b2f7-9f06-4a6f-b57f-f41068adfa81 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:48.586443 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8b2f7-9f06-4a6f-b57f-f41068adfa81 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:48.588107 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 693/1377] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:48.594773 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb75357-5130-41d0-ab36-2b1ef069a8a2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:48.595324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb75357-5130-41d0-ab36-2b1ef069a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:48.595449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb75357-5130-41d0-ab36-2b1ef069a8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:48.677354 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-efb75357-5130-41d0-ab36-2b1ef069a8a2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:48.678772 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-efb75357-5130-41d0-ab36-2b1ef069a8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:48.686380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:48.815683 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 685/1378] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:49.321549 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8b2f7-9f06-4a6f-b57f-f41068adfa81 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:49.323327 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-c0c8b2f7-9f06-4a6f-b57f-f41068adfa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:49.428178 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:49.556933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8b2f7-9f06-4a6f-b57f-f41068adfa81 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:49.698265 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0910cb-6c8a-443f-9e83-4b6b826c2830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:49.699252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0910cb-6c8a-443f-9e83-4b6b826c2830 service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:49.699252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0910cb-6c8a-443f-9e83-4b6b826c2830 service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:49.699518 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0910cb-6c8a-443f-9e83-4b6b826c2830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:49.701123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0b0910cb-6c8a-443f-9e83-4b6b826c2830 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:49.722368 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:49.805155 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0910cb-6c8a-443f-9e83-4b6b826c2830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:49.806837 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0910cb-6c8a-443f-9e83-4b6b826c2830 service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:49.830180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 694/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 12:11:52.047151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e7c98-3d52-4c06-b023-88a265c47ec8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:52.047151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e7c98-3d52-4c06-b023-88a265c47ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:52.047151 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e7c98-3d52-4c06-b023-88a265c47ec8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:52.053454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 686/1380] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:52.054842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17040402-2458-49e3-999a-58b4b1ae7678 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:52.054842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17040402-2458-49e3-999a-58b4b1ae7678 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:52.054842 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17040402-2458-49e3-999a-58b4b1ae7678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:52.130719 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-17040402-2458-49e3-999a-58b4b1ae7678 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:52.131959 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-17040402-2458-49e3-999a-58b4b1ae7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:52.243734 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:52.251288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 695/1381] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:11:52.560337 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e7c98-3d52-4c06-b023-88a265c47ec8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:52.562955 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-710e7c98-3d52-4c06-b023-88a265c47ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:52.778369 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:52.791596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e7c98-3d52-4c06-b023-88a265c47ec8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:52.964819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a193c-ca36-4f15-b7c9-f8177770d283 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:52.965500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a193c-ca36-4f15-b7c9-f8177770d283 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:52.965886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a193c-ca36-4f15-b7c9-f8177770d283 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:52.966171 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a193c-ca36-4f15-b7c9-f8177770d283 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:52.971329 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0c0a193c-ca36-4f15-b7c9-f8177770d283 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:53.112481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a193c-ca36-4f15-b7c9-f8177770d283 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:53.114174 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a193c-ca36-4f15-b7c9-f8177770d283 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:53.132140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:53.145641 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 687/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:55.473131 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3db841-0bae-42ad-a971-2b0cd8ac404f None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:55.473702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3db841-0bae-42ad-a971-2b0cd8ac404f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:55.473702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3db841-0bae-42ad-a971-2b0cd8ac404f None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:55.475100 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 696/1383] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:55.480456 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed39cad-e734-483f-9eea-e807b2339c22 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:55.481069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed39cad-e734-483f-9eea-e807b2339c22 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:55.481069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed39cad-e734-483f-9eea-e807b2339c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:55.565754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-eed39cad-e734-483f-9eea-e807b2339c22 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:55.566993 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-eed39cad-e734-483f-9eea-e807b2339c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:55.729198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 688/1384] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:11:56.087815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3db841-0bae-42ad-a971-2b0cd8ac404f None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:56.089815 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-4c3db841-0bae-42ad-a971-2b0cd8ac404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:56.303767 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3db841-0bae-42ad-a971-2b0cd8ac404f None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:11:56.407333 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:56.500734 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badf6cf-8671-46ce-8796-7d4da3c68c76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:11:56.502057 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf6cf-8671-46ce-8796-7d4da3c68c76 service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:56.502169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf6cf-8671-46ce-8796-7d4da3c68c76 service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:56.502517 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badf6cf-8671-46ce-8796-7d4da3c68c76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:56.504568 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-8badf6cf-8671-46ce-8796-7d4da3c68c76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:56.633161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badf6cf-8671-46ce-8796-7d4da3c68c76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:11:56.635924 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badf6cf-8671-46ce-8796-7d4da3c68c76 service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:11:56.665555 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 697/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:11:59.195954 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b53b84-4867-405d-b751-7ffaa911c1a8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:59.196563 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b53b84-4867-405d-b751-7ffaa911c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:59.196676 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b53b84-4867-405d-b751-7ffaa911c1a8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:59.198322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 689/1386] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:11:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:11:59.205175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246de34-5ffa-4d61-ada1-b6003ac8a832 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:11:59.205688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246de34-5ffa-4d61-ada1-b6003ac8a832 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:11:59.205806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246de34-5ffa-4d61-ada1-b6003ac8a832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:11:59.279788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-a246de34-5ffa-4d61-ada1-b6003ac8a832 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:11:59.281969 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-a246de34-5ffa-4d61-ada1-b6003ac8a832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:59.320588 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:59.392064 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 698/1387] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:11:59.698946 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b53b84-4867-405d-b751-7ffaa911c1a8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:11:59.700938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-10b53b84-4867-405d-b751-7ffaa911c1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:11:59.798340 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:11:59.881810 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b53b84-4867-405d-b751-7ffaa911c1a8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:00.023851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95b4bb-612f-4cea-a165-fd2f95db4e23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:00.024711 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95b4bb-612f-4cea-a165-fd2f95db4e23 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:00.025094 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95b4bb-612f-4cea-a165-fd2f95db4e23 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:00.025446 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95b4bb-612f-4cea-a165-fd2f95db4e23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:00.027454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0b95b4bb-612f-4cea-a165-fd2f95db4e23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:00.073106 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:00.129538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95b4bb-612f-4cea-a165-fd2f95db4e23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:00.131226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95b4bb-612f-4cea-a165-fd2f95db4e23 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:00.150834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 690/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:02.273339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8ace-9d33-42f6-ad02-e9ecb2937fb1 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:02.274088 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8ace-9d33-42f6-ad02-e9ecb2937fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:02.274088 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8ace-9d33-42f6-ad02-e9ecb2937fb1 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:02.275572 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 699/1389] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 12:12:02.284791 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:02.286223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:02.286782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:02.361997 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:02.363273 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:02.427852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62339-b239-423f-bab0-b9e79345e97e None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:02.428119 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62339-b239-423f-bab0-b9e79345e97e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:02.428395 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc62339-b239-423f-bab0-b9e79345e97e None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:02.430062 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 700/1390] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 12:12:02.435255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535d00a-572c-4192-9b2e-3519248da111 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:02.435537 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535d00a-572c-4192-9b2e-3519248da111 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:02.435819 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535d00a-572c-4192-9b2e-3519248da111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:02.474738 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 691/1391] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:02.507649 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2535d00a-572c-4192-9b2e-3519248da111 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:02.508867 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2535d00a-572c-4192-9b2e-3519248da111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:02.608041 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 701/1392] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:12:02.768598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:02.769851 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:02.786401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:02.911227 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca54893-1772-4452-9b70-c45dc9dec4eb None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:02.914083 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e347e-7340-4d73-a874-1e224ba8d3b2 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:02.914379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e347e-7340-4d73-a874-1e224ba8d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:02.914664 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e347e-7340-4d73-a874-1e224ba8d3b2 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:02.916027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 702/1393] 10.222.0.48 () {54 vars in 967 bytes} [Thu Sep 1 12:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 12:12:02.921354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417157f-8b6f-4cd9-955a-e535bf139ddd None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:02.921673 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417157f-8b6f-4cd9-955a-e535bf139ddd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:02.921961 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417157f-8b6f-4cd9-955a-e535bf139ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:02.938104 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:03.008292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-e417157f-8b6f-4cd9-955a-e535bf139ddd None None] MFA Rules not processed for user `bb79770a161740128e5792a70cbad00a`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:03.009596 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-e417157f-8b6f-4cd9-955a-e535bf139ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:03.073628 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8bc32-ad50-4f04-908a-93c478df36c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:03.075370 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8bc32-ad50-4f04-908a-93c478df36c8 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:03.076040 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8bc32-ad50-4f04-908a-93c478df36c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:03.076394 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8bc32-ad50-4f04-908a-93c478df36c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:03.079249 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a0e8bc32-ad50-4f04-908a-93c478df36c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:03.091126 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:03.091447 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 703/1394] 10.222.0.48 () {58 vars in 1064 bytes} [Thu Sep 1 12:12:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:12:03.099807 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417157f-8b6f-4cd9-955a-e535bf139ddd None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:03.108109 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417157f-8b6f-4cd9-955a-e535bf139ddd None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:03.185660 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cf63b-759e-442c-b23c-5039d5c34ea1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb79770a161740128e5792a70cbad00a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:03.186201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf63b-759e-442c-b23c-5039d5c34ea1 service glance] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:03.186998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf63b-759e-442c-b23c-5039d5c34ea1 service glance] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:03.187279 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cf63b-759e-442c-b23c-5039d5c34ea1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:03.188931 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-400cf63b-759e-442c-b23c-5039d5c34ea1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:03.198588 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8bc32-ad50-4f04-908a-93c478df36c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:03.201462 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8bc32-ad50-4f04-908a-93c478df36c8 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:03.222217 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:03.236401 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 692/1395] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 12:12:03.334117 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf63b-759e-442c-b23c-5039d5c34ea1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:03.335606 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cf63b-759e-442c-b23c-5039d5c34ea1 service glance] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:03.357886 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 704/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 12:12:04.319198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7669-c491-4021-bd6d-cb41ba9e24a9 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:04.319198 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7669-c491-4021-bd6d-cb41ba9e24a9 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:04.320200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7669-c491-4021-bd6d-cb41ba9e24a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:04.405991 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-701b7669-c491-4021-bd6d-cb41ba9e24a9 None None] MFA Rules not processed for user `4e0eeaf94d9749c7b0dc15a97a3bf81a`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:04.407050 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-701b7669-c491-4021-bd6d-cb41ba9e24a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:04.508417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:04.512370 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 693/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Sep 1 12:12:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:04.562829 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139e704-d8c8-49cd-b3c4-8c5be095baf6 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:04.563412 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139e704-d8c8-49cd-b3c4-8c5be095baf6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:04.563412 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139e704-d8c8-49cd-b3c4-8c5be095baf6 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:04.564747 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 705/1398] 10.222.0.48 () {54 vars in 943 bytes} [Thu Sep 1 12:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:04.571864 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cb61-78b9-491b-8035-490d501d3635 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:04.572154 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cb61-78b9-491b-8035-490d501d3635 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:04.572430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cb61-78b9-491b-8035-490d501d3635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:04.660386 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-ad45cb61-78b9-491b-8035-490d501d3635 None None] MFA Rules not processed for user `60c4eccc7fb0434e9251fa1b9bd7aec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:04.661521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-ad45cb61-78b9-491b-8035-490d501d3635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:04.782604 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 694/1399] 10.222.0.48 () {58 vars in 1040 bytes} [Thu Sep 1 12:12:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:04.792260 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139e704-d8c8-49cd-b3c4-8c5be095baf6 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:04.794099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-c139e704-d8c8-49cd-b3c4-8c5be095baf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:04.933871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:04.936752 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139e704-d8c8-49cd-b3c4-8c5be095baf6 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:05.051484 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36621b8c-862a-44c1-bdbf-43562afcfc16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c4eccc7fb0434e9251fa1b9bd7aec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:05.052112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36621b8c-862a-44c1-bdbf-43562afcfc16 service swift] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:05.052392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36621b8c-862a-44c1-bdbf-43562afcfc16 service swift] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:05.052669 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36621b8c-862a-44c1-bdbf-43562afcfc16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:05.054684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-36621b8c-862a-44c1-bdbf-43562afcfc16 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:05.141661 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36621b8c-862a-44c1-bdbf-43562afcfc16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:05.143288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36621b8c-862a-44c1-bdbf-43562afcfc16 service swift] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:05.151649 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 706/1400] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Sep 1 12:12:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 12:12:05.696175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5526-953f-4832-8574-c7047a1da06e None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:05.696686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5526-953f-4832-8574-c7047a1da06e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:05.697104 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5526-953f-4832-8574-c7047a1da06e None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:05.700276 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 695/1401] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:12:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:05.706392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe303c-73bc-4b63-aaff-5f8357540a1c None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:05.707323 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe303c-73bc-4b63-aaff-5f8357540a1c None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:05.707798 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe303c-73bc-4b63-aaff-5f8357540a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:05.772744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:05.818925 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-effe303c-73bc-4b63-aaff-5f8357540a1c None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:05.820171 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-effe303c-73bc-4b63-aaff-5f8357540a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:05.953094 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 707/1402] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:06.243554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5526-953f-4832-8574-c7047a1da06e None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:06.245176 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-160a5526-953f-4832-8574-c7047a1da06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:06.328687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:06.433656 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5526-953f-4832-8574-c7047a1da06e None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:06.596502 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0454ab-6921-49e7-bab9-9b3de87ad66e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:06.597510 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0454ab-6921-49e7-bab9-9b3de87ad66e service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:06.597825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0454ab-6921-49e7-bab9-9b3de87ad66e service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:06.598095 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0454ab-6921-49e7-bab9-9b3de87ad66e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:06.599794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9a0454ab-6921-49e7-bab9-9b3de87ad66e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:06.627435 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:06.723523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0454ab-6921-49e7-bab9-9b3de87ad66e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:06.726003 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0454ab-6921-49e7-bab9-9b3de87ad66e service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:06.747574 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 696/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:08.974066 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1858-5e5f-4b5d-ad99-5ce93f4c4212 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:08.974677 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1858-5e5f-4b5d-ad99-5ce93f4c4212 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:08.974677 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1858-5e5f-4b5d-ad99-5ce93f4c4212 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:08.976259 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 708/1404] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:08.981543 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698efc3-798b-44d2-880c-fabf58c66ca1 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:08.982136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698efc3-798b-44d2-880c-fabf58c66ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:08.982136 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698efc3-798b-44d2-880c-fabf58c66ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:09.064010 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-b698efc3-798b-44d2-880c-fabf58c66ca1 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:09.065244 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b698efc3-798b-44d2-880c-fabf58c66ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:09.165660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:09.172589 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 697/1405] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:09.466367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1858-5e5f-4b5d-ad99-5ce93f4c4212 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:09.467976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-e58f1858-5e5f-4b5d-ad99-5ce93f4c4212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:09.617856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:09.670689 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1858-5e5f-4b5d-ad99-5ce93f4c4212 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:09.817421 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06039d-bacc-4fcf-b893-2203e6512a0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:09.818208 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06039d-bacc-4fcf-b893-2203e6512a0f service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:09.818541 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06039d-bacc-4fcf-b893-2203e6512a0f service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:09.818878 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06039d-bacc-4fcf-b893-2203e6512a0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:09.821980 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-6e06039d-bacc-4fcf-b893-2203e6512a0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:09.909659 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:09.931289 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06039d-bacc-4fcf-b893-2203e6512a0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:09.933902 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06039d-bacc-4fcf-b893-2203e6512a0f service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:09.961956 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 709/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:12.441818 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e29ad-5776-46b3-8aa7-3b519ae4f3a5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:12.442406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e29ad-5776-46b3-8aa7-3b519ae4f3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:12.442406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e29ad-5776-46b3-8aa7-3b519ae4f3a5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:12.448679 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 698/1407] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:12.450728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d070407-b220-4bcc-9f37-fe5e04a2506e None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:12.451200 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d070407-b220-4bcc-9f37-fe5e04a2506e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:12.451304 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d070407-b220-4bcc-9f37-fe5e04a2506e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:12.550752 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-3d070407-b220-4bcc-9f37-fe5e04a2506e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:12.552165 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3d070407-b220-4bcc-9f37-fe5e04a2506e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:12.701009 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 710/1408] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:13.164879 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e29ad-5776-46b3-8aa7-3b519ae4f3a5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:13.169107 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3f4e29ad-5776-46b3-8aa7-3b519ae4f3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:13.445529 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e29ad-5776-46b3-8aa7-3b519ae4f3a5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:13.551223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:13.634680 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5d67d-0a43-47b0-9022-8f50f24ff8d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:13.736735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d67d-0a43-47b0-9022-8f50f24ff8d4 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:13.736735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d67d-0a43-47b0-9022-8f50f24ff8d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:13.736735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d67d-0a43-47b0-9022-8f50f24ff8d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:13.736735 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-adf5d67d-0a43-47b0-9022-8f50f24ff8d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:13.806449 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5d67d-0a43-47b0-9022-8f50f24ff8d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:13.808737 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5d67d-0a43-47b0-9022-8f50f24ff8d4 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:13.837987 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 699/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:22.698316 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=79 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:24.232649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=64 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:30.741737 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993218-8baa-47fb-9bff-52c194f79bfd None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:30.742314 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993218-8baa-47fb-9bff-52c194f79bfd None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:30.742314 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993218-8baa-47fb-9bff-52c194f79bfd None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:30.743707 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 711/1410] 10.222.0.48 () {54 vars in 918 bytes} [Thu Sep 1 12:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:30.749223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045980d-2328-4ea7-add1-5fc7b71ea235 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:30.749774 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045980d-2328-4ea7-add1-5fc7b71ea235 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:30.749774 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045980d-2328-4ea7-add1-5fc7b71ea235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:30.831843 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5045980d-2328-4ea7-add1-5fc7b71ea235 None None] MFA Rules not processed for user `72ca589ac0054417b433852ec38b17a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:30.833120 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5045980d-2328-4ea7-add1-5fc7b71ea235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:30.948078 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 700/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Sep 1 12:12:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:30.963005 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0bc4d-8bf0-47ba-96cd-9f76010d4888 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:30.963318 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0bc4d-8bf0-47ba-96cd-9f76010d4888 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:30.963600 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0bc4d-8bf0-47ba-96cd-9f76010d4888 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:30.964964 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 712/1412] 10.222.0.48 () {54 vars in 918 bytes} [Thu Sep 1 12:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:30.970718 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727d0e8-3de4-429e-a084-cabeb5760a4b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:30.971003 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727d0e8-3de4-429e-a084-cabeb5760a4b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:30.971278 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727d0e8-3de4-429e-a084-cabeb5760a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:31.052430 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:31.058302 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-f727d0e8-3de4-429e-a084-cabeb5760a4b None None] MFA Rules not processed for user `0c8225879a66415c8a1dc28584e4c626`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:31.059530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f727d0e8-3de4-429e-a084-cabeb5760a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:31.183412 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 701/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Sep 1 12:12:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:33.563248 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f047267-2667-420d-a71d-4d020b1c2e88 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:33.563812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f047267-2667-420d-a71d-4d020b1c2e88 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:33.563812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f047267-2667-420d-a71d-4d020b1c2e88 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:33.565156 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:33.570763 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ec23-2ff1-4958-8f74-2ce19d6b5b67 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:33.571312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ec23-2ff1-4958-8f74-2ce19d6b5b67 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:33.571312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ec23-2ff1-4958-8f74-2ce19d6b5b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:33.644470 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-6472ec23-2ff1-4958-8f74-2ce19d6b5b67 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:33.645739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6472ec23-2ff1-4958-8f74-2ce19d6b5b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:33.707110 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:33.752530 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:34.043535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f047267-2667-420d-a71d-4d020b1c2e88 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:34.045096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-4f047267-2667-420d-a71d-4d020b1c2e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:34.203567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f047267-2667-420d-a71d-4d020b1c2e88 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:34.276641 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:34.318750 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34181e9-6b2b-43a0-9130-f2a91834c303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:34.319295 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34181e9-6b2b-43a0-9130-f2a91834c303 service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:34.319575 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34181e9-6b2b-43a0-9130-f2a91834c303 service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:34.319847 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34181e9-6b2b-43a0-9130-f2a91834c303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:34.321464 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-e34181e9-6b2b-43a0-9130-f2a91834c303 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:34.420292 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34181e9-6b2b-43a0-9130-f2a91834c303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:34.422564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34181e9-6b2b-43a0-9130-f2a91834c303 service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:34.441593 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 714/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:35.348905 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77973b5-7e1a-4785-977c-ce0bebd96356 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:35.349484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77973b5-7e1a-4785-977c-ce0bebd96356 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:35.349484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77973b5-7e1a-4785-977c-ce0bebd96356 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:35.350834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:35.355860 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0855-449a-4f6c-bed6-587b80f50992 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:35.356299 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0855-449a-4f6c-bed6-587b80f50992 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:35.356427 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0855-449a-4f6c-bed6-587b80f50992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:35.437061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2c2b0855-449a-4f6c-bed6-587b80f50992 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:35.438367 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2c2b0855-449a-4f6c-bed6-587b80f50992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:35.469095 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:35.539971 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:35.693675 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77973b5-7e1a-4785-977c-ce0bebd96356 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:35.695290 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b77973b5-7e1a-4785-977c-ce0bebd96356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:35.868780 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77973b5-7e1a-4785-977c-ce0bebd96356 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:35.915576 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:35.998423 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b106e-3bc2-484b-877c-a8b0a49ca6da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:35.999111 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b106e-3bc2-484b-877c-a8b0a49ca6da service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:35.999392 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b106e-3bc2-484b-877c-a8b0a49ca6da service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:35.999664 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b106e-3bc2-484b-877c-a8b0a49ca6da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:36.001350 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-991b106e-3bc2-484b-877c-a8b0a49ca6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:36.096732 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b106e-3bc2-484b-877c-a8b0a49ca6da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:36.098387 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b106e-3bc2-484b-877c-a8b0a49ca6da service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:36.117158 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 704/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:37.003469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed307a6-c9a8-4d60-8797-d79bb8a67d5a None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:37.004023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed307a6-c9a8-4d60-8797-d79bb8a67d5a None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:37.004023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed307a6-c9a8-4d60-8797-d79bb8a67d5a None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:37.005429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:37.010497 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc4eb0-85d1-4ea0-9391-925ec5b3aadf None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:37.010910 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc4eb0-85d1-4ea0-9391-925ec5b3aadf None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:37.011025 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc4eb0-85d1-4ea0-9391-925ec5b3aadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:37.065315 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:37.086623 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-73dc4eb0-85d1-4ea0-9391-925ec5b3aadf None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:37.087834 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-73dc4eb0-85d1-4ea0-9391-925ec5b3aadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:37.182546 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:37.333768 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed307a6-c9a8-4d60-8797-d79bb8a67d5a None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:37.335229 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2ed307a6-c9a8-4d60-8797-d79bb8a67d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:37.486939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed307a6-c9a8-4d60-8797-d79bb8a67d5a None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:37.499450 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:37.604740 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40d386-df9f-4684-9c12-1204c6faf33f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:37.605263 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40d386-df9f-4684-9c12-1204c6faf33f service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:37.605542 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40d386-df9f-4684-9c12-1204c6faf33f service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:37.605844 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40d386-df9f-4684-9c12-1204c6faf33f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:37.607472 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-8c40d386-df9f-4684-9c12-1204c6faf33f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:37.698491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40d386-df9f-4684-9c12-1204c6faf33f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:37.700076 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40d386-df9f-4684-9c12-1204c6faf33f service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:37.719946 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 717/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:38.586837 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120846-6bb4-405c-bf95-a3ee0ec022dc None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:38.587415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120846-6bb4-405c-bf95-a3ee0ec022dc None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:38.587415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120846-6bb4-405c-bf95-a3ee0ec022dc None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:38.588782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:38.594442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516b15e-82cc-4b03-9a18-e8b6bb401034 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:38.594964 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516b15e-82cc-4b03-9a18-e8b6bb401034 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:38.595061 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516b15e-82cc-4b03-9a18-e8b6bb401034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:38.615305 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:38.798973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-2516b15e-82cc-4b03-9a18-e8b6bb401034 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:38.800220 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-2516b15e-82cc-4b03-9a18-e8b6bb401034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:38.902684 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:39.050962 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f120846-6bb4-405c-bf95-a3ee0ec022dc None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:39.052406 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8f120846-6bb4-405c-bf95-a3ee0ec022dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:39.161263 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:39.199211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f120846-6bb4-405c-bf95-a3ee0ec022dc None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:39.314637 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2674a-db55-4ecc-8cbe-bc3c1ffb2f24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:39.315160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2674a-db55-4ecc-8cbe-bc3c1ffb2f24 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:39.315436 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2674a-db55-4ecc-8cbe-bc3c1ffb2f24 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:39.315702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2674a-db55-4ecc-8cbe-bc3c1ffb2f24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:39.317312 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6dd2674a-db55-4ecc-8cbe-bc3c1ffb2f24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:39.398671 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:39.412366 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2674a-db55-4ecc-8cbe-bc3c1ffb2f24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:39.415555 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2674a-db55-4ecc-8cbe-bc3c1ffb2f24 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:39.433500 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 707/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:40.312933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e7b03-0466-416a-bafa-97c0eae2f42c None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:40.313491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e7b03-0466-416a-bafa-97c0eae2f42c None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:40.313491 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e7b03-0466-416a-bafa-97c0eae2f42c None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:40.314856 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:40.319949 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa3b65-b3df-4fdc-b0df-dd00ee297b8c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:40.320415 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa3b65-b3df-4fdc-b0df-dd00ee297b8c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:40.320521 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa3b65-b3df-4fdc-b0df-dd00ee297b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:40.404437 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5baa3b65-b3df-4fdc-b0df-dd00ee297b8c None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:40.405746 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5baa3b65-b3df-4fdc-b0df-dd00ee297b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:40.521795 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:40.671065 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e7b03-0466-416a-bafa-97c0eae2f42c None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:40.672555 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-831e7b03-0466-416a-bafa-97c0eae2f42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:40.758141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:40.822099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e7b03-0466-416a-bafa-97c0eae2f42c None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:40.933950 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6eddf-6743-4116-9ee7-8acb42f8b60e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:40.934442 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6eddf-6743-4116-9ee7-8acb42f8b60e service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:40.934714 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6eddf-6743-4116-9ee7-8acb42f8b60e service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:40.934982 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6eddf-6743-4116-9ee7-8acb42f8b60e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:40.937132 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9bf6eddf-6743-4116-9ee7-8acb42f8b60e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:40.986832 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:41.028619 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6eddf-6743-4116-9ee7-8acb42f8b60e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:41.030727 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6eddf-6743-4116-9ee7-8acb42f8b60e service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:41.049563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 720/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:41.921289 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef905cd-021d-4590-b5a7-c9129cc75450 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:41.921896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef905cd-021d-4590-b5a7-c9129cc75450 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:41.921896 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef905cd-021d-4590-b5a7-c9129cc75450 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:41.923203 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:41.928788 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c5e6-8c8a-400c-8e77-41ebe804dae6 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:41.929238 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c5e6-8c8a-400c-8e77-41ebe804dae6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:41.929429 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c5e6-8c8a-400c-8e77-41ebe804dae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:42.007087 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-13a9c5e6-8c8a-400c-8e77-41ebe804dae6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:42.008317 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-13a9c5e6-8c8a-400c-8e77-41ebe804dae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:42.113987 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:42.267122 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef905cd-021d-4590-b5a7-c9129cc75450 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:42.268686 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7ef905cd-021d-4590-b5a7-c9129cc75450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:42.318564 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:42.415035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef905cd-021d-4590-b5a7-c9129cc75450 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:42.531902 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074870dc-882a-4c71-b1a6-e8ff0e0b63f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:42.532422 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074870dc-882a-4c71-b1a6-e8ff0e0b63f2 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:42.532706 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074870dc-882a-4c71-b1a6-e8ff0e0b63f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:42.532979 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074870dc-882a-4c71-b1a6-e8ff0e0b63f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:42.534649 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-074870dc-882a-4c71-b1a6-e8ff0e0b63f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:42.541899 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:42.626535 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074870dc-882a-4c71-b1a6-e8ff0e0b63f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:42.628158 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074870dc-882a-4c71-b1a6-e8ff0e0b63f2 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:42.645964 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 710/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:43.510286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0913be-3cf5-4cf3-b673-9c6af9f26935 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:43.510863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0913be-3cf5-4cf3-b673-9c6af9f26935 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:43.510863 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0913be-3cf5-4cf3-b673-9c6af9f26935 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:43.512183 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:43.518200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0dff5-a5dd-45e3-9d59-35e8c5dde736 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:43.518687 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0dff5-a5dd-45e3-9d59-35e8c5dde736 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:43.518784 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0dff5-a5dd-45e3-9d59-35e8c5dde736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:43.599723 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-21d0dff5-a5dd-45e3-9d59-35e8c5dde736 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:43.603405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-21d0dff5-a5dd-45e3-9d59-35e8c5dde736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:43.670087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:43.709322 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:43.862658 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0913be-3cf5-4cf3-b673-9c6af9f26935 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:43.864186 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9f0913be-3cf5-4cf3-b673-9c6af9f26935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:43.882460 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:44.009939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0913be-3cf5-4cf3-b673-9c6af9f26935 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:44.096345 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:44.123627 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68336-9875-4096-9ce0-b41ef76b3a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:44.124230 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68336-9875-4096-9ce0-b41ef76b3a02 service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:44.125236 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68336-9875-4096-9ce0-b41ef76b3a02 service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:44.125236 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68336-9875-4096-9ce0-b41ef76b3a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:44.126555 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-16e68336-9875-4096-9ce0-b41ef76b3a02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:44.219233 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e68336-9875-4096-9ce0-b41ef76b3a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:44.220900 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e68336-9875-4096-9ce0-b41ef76b3a02 service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:44.239870 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 723/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:45.102481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b151-fba6-4d78-ac8e-c6b8bb62887a None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:45.103085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b151-fba6-4d78-ac8e-c6b8bb62887a None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:45.103085 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b151-fba6-4d78-ac8e-c6b8bb62887a None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:45.104424 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:45.109800 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c715f-050e-450a-9cae-7d338ed09e04 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:45.110594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c715f-050e-450a-9cae-7d338ed09e04 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:45.110594 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c715f-050e-450a-9cae-7d338ed09e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:45.188879 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-ff8c715f-050e-450a-9cae-7d338ed09e04 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:45.190161 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-ff8c715f-050e-450a-9cae-7d338ed09e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:45.231111 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:45.286634 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:45.438533 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b151-fba6-4d78-ac8e-c6b8bb62887a None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:45.440016 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d1c6b151-fba6-4d78-ac8e-c6b8bb62887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:45.587874 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b151-fba6-4d78-ac8e-c6b8bb62887a None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:45.641645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:45.701283 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b6093-d304-432d-9de6-4442afd2dcd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:45.701868 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b6093-d304-432d-9de6-4442afd2dcd5 service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:45.702148 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b6093-d304-432d-9de6-4442afd2dcd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:45.702413 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b6093-d304-432d-9de6-4442afd2dcd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:45.704033 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fd4b6093-d304-432d-9de6-4442afd2dcd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:45.797069 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b6093-d304-432d-9de6-4442afd2dcd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:45.798742 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b6093-d304-432d-9de6-4442afd2dcd5 service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:45.817351 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 713/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:46.692981 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8529-7d34-4748-9f0b-23423f9e6d72 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:46.693570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8529-7d34-4748-9f0b-23423f9e6d72 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:46.693570 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8529-7d34-4748-9f0b-23423f9e6d72 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:46.694917 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 12:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:46.700079 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2a93-e54e-41d2-a510-0021a3a5cce6 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:46.700618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2a93-e54e-41d2-a510-0021a3a5cce6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:46.700618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2a93-e54e-41d2-a510-0021a3a5cce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:46.778969 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-3d2f2a93-e54e-41d2-a510-0021a3a5cce6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:46.780211 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-3d2f2a93-e54e-41d2-a510-0021a3a5cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:46.782348 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:46.879608 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 12:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:47.031882 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c8529-7d34-4748-9f0b-23423f9e6d72 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:47.033416 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-026c8529-7d34-4748-9f0b-23423f9e6d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:47.179836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c8529-7d34-4748-9f0b-23423f9e6d72 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:47.206154 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:47.291307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988802d1-d3e6-4e1e-a673-f6b3b80b62cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:47.291821 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988802d1-d3e6-4e1e-a673-f6b3b80b62cd service nova] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:47.292092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988802d1-d3e6-4e1e-a673-f6b3b80b62cd service nova] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:47.292357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988802d1-d3e6-4e1e-a673-f6b3b80b62cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:47.294038 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-988802d1-d3e6-4e1e-a673-f6b3b80b62cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:47.384244 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988802d1-d3e6-4e1e-a673-f6b3b80b62cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:47.385881 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988802d1-d3e6-4e1e-a673-f6b3b80b62cd service nova] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:47.404372 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 726/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:49.230328 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd44b4-5d50-4903-9025-872c9d0ec2da None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:49.230921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd44b4-5d50-4903-9025-872c9d0ec2da None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:49.230921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd44b4-5d50-4903-9025-872c9d0ec2da None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:49.232292 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:49.238099 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c6552-2f76-4523-8543-94ef572d5741 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:49.238600 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c6552-2f76-4523-8543-94ef572d5741 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:49.238715 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c6552-2f76-4523-8543-94ef572d5741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:49.279320 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:49.313636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-5f2c6552-2f76-4523-8543-94ef572d5741 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:49.314859 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-5f2c6552-2f76-4523-8543-94ef572d5741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:49.416514 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:12:49.702774 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd44b4-5d50-4903-9025-872c9d0ec2da None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:49.704358 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b4fd44b4-5d50-4903-9025-872c9d0ec2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:49.816624 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd44b4-5d50-4903-9025-872c9d0ec2da None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:49.850532 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:49.895771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8c414-e174-47ee-958b-87981b304a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:49.896380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c414-e174-47ee-958b-87981b304a23 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:49.896660 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c414-e174-47ee-958b-87981b304a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:49.896932 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c414-e174-47ee-958b-87981b304a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:49.898592 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-55a8c414-e174-47ee-958b-87981b304a23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:50.019603 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8c414-e174-47ee-958b-87981b304a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:50.021170 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8c414-e174-47ee-958b-87981b304a23 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:50.040453 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 716/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:52.115297 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cbf0c-22ad-42b6-bf30-715fcfa9165f None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:52.115890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cbf0c-22ad-42b6-bf30-715fcfa9165f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:52.115890 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cbf0c-22ad-42b6-bf30-715fcfa9165f None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:52.117201 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:52.122596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16641e-d0bc-42ba-a124-6c25dcde3b9b None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:52.123096 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16641e-d0bc-42ba-a124-6c25dcde3b9b None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:52.123201 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16641e-d0bc-42ba-a124-6c25dcde3b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:52.190277 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-5e16641e-d0bc-42ba-a124-6c25dcde3b9b None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:52.191510 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5e16641e-d0bc-42ba-a124-6c25dcde3b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:52.204484 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:52.289075 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:52.574769 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cbf0c-22ad-42b6-bf30-715fcfa9165f None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:52.576388 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-794cbf0c-22ad-42b6-bf30-715fcfa9165f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:52.684338 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cbf0c-22ad-42b6-bf30-715fcfa9165f None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:52.696696 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:52.757810 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb321d5-34de-4cac-9c48-235545a17ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:52.758408 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb321d5-34de-4cac-9c48-235545a17ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:52.758683 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb321d5-34de-4cac-9c48-235545a17ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:52.758951 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb321d5-34de-4cac-9c48-235545a17ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:52.760543 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0fb321d5-34de-4cac-9c48-235545a17ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:52.867757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb321d5-34de-4cac-9c48-235545a17ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:52.869174 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb321d5-34de-4cac-9c48-235545a17ca6 service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:52.894806 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 729/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:55.214041 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f131-1ec2-4cd4-bf2e-a2cae26b5eb5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:55.214632 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f131-1ec2-4cd4-bf2e-a2cae26b5eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:55.214632 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f131-1ec2-4cd4-bf2e-a2cae26b5eb5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:55.215936 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:55.221277 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a139bb-a3d2-46a2-81ef-f77381a9e762 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:55.221722 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a139bb-a3d2-46a2-81ef-f77381a9e762 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:55.221871 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a139bb-a3d2-46a2-81ef-f77381a9e762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:55.270072 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:55.288307 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-51a139bb-a3d2-46a2-81ef-f77381a9e762 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:55.289535 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-51a139bb-a3d2-46a2-81ef-f77381a9e762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:55.385390 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:55.668373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f131-1ec2-4cd4-bf2e-a2cae26b5eb5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:55.669962 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-fd81f131-1ec2-4cd4-bf2e-a2cae26b5eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:55.774808 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f131-1ec2-4cd4-bf2e-a2cae26b5eb5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:55.818044 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:55.852438 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54538bf-e514-474b-86ad-0eae98f04b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:55.853048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54538bf-e514-474b-86ad-0eae98f04b0a service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:55.853324 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54538bf-e514-474b-86ad-0eae98f04b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:55.853595 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54538bf-e514-474b-86ad-0eae98f04b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:55.855265 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b54538bf-e514-474b-86ad-0eae98f04b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:55.966126 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54538bf-e514-474b-86ad-0eae98f04b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:55.968223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54538bf-e514-474b-86ad-0eae98f04b0a service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:55.986369 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 719/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:12:58.054334 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1685-d7bd-4ae2-bd62-b790aafe0cf6 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:58.055080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1685-d7bd-4ae2-bd62-b790aafe0cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:58.055080 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1685-d7bd-4ae2-bd62-b790aafe0cf6 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:58.056242 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:12:58.062024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4e9b4-947e-499b-b6e2-260d644b0a65 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:58.062554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4e9b4-947e-499b-b6e2-260d644b0a65 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:58.062554 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4e9b4-947e-499b-b6e2-260d644b0a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:58.135753 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-03a4e9b4-947e-499b-b6e2-260d644b0a65 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:12:58.136996 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-03a4e9b4-947e-499b-b6e2-260d644b0a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:58.168314 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:58.259816 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:12:58.544157 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1685-d7bd-4ae2-bd62-b790aafe0cf6 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:12:58.545770 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3fef1685-d7bd-4ae2-bd62-b790aafe0cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:58.651645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1685-d7bd-4ae2-bd62-b790aafe0cf6 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:12:58.673066 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:12:58.727545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48445-dd1a-4037-af75-8f907ef3ec2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:12:58.728159 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48445-dd1a-4037-af75-8f907ef3ec2f service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:12:58.728435 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48445-dd1a-4037-af75-8f907ef3ec2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:12:58.728702 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48445-dd1a-4037-af75-8f907ef3ec2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:12:58.730339 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-49b48445-dd1a-4037-af75-8f907ef3ec2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:12:58.841446 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48445-dd1a-4037-af75-8f907ef3ec2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:12:58.842923 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48445-dd1a-4037-af75-8f907ef3ec2f service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:12:58.861623 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 732/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:00.926194 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24561d30-cf6f-4f73-ad1d-cc560823e7f6 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:00.926770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24561d30-cf6f-4f73-ad1d-cc560823e7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:00.926770 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24561d30-cf6f-4f73-ad1d-cc560823e7f6 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:00.928076 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:00.933691 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08968b9-8abd-42f2-8e19-d2d316fc27d6 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:00.934137 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08968b9-8abd-42f2-8e19-d2d316fc27d6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:00.934255 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08968b9-8abd-42f2-8e19-d2d316fc27d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:00.999836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:01.002603 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-f08968b9-8abd-42f2-8e19-d2d316fc27d6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:01.003845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-f08968b9-8abd-42f2-8e19-d2d316fc27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:01.103081 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:01.396341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24561d30-cf6f-4f73-ad1d-cc560823e7f6 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:01.397924 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-24561d30-cf6f-4f73-ad1d-cc560823e7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:01.506139 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24561d30-cf6f-4f73-ad1d-cc560823e7f6 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:01.556939 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:01.583729 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de49be-e82e-49a3-99d1-715559d6e888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:01.584341 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de49be-e82e-49a3-99d1-715559d6e888 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:01.584618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de49be-e82e-49a3-99d1-715559d6e888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:01.584886 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de49be-e82e-49a3-99d1-715559d6e888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:01.586552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-f3de49be-e82e-49a3-99d1-715559d6e888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:01.701043 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de49be-e82e-49a3-99d1-715559d6e888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:01.702479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de49be-e82e-49a3-99d1-715559d6e888 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:01.720597 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 722/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:03.781582 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8e2f4-833e-4eb6-8e1e-01737c1deb9b None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:03.782175 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8e2f4-833e-4eb6-8e1e-01737c1deb9b None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:03.782296 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8e2f4-833e-4eb6-8e1e-01737c1deb9b None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:03.783637 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:13:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:03.789138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194dc0f-5a71-4aa3-8fda-bac0f5739a2c None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:03.789591 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194dc0f-5a71-4aa3-8fda-bac0f5739a2c None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:03.789731 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194dc0f-5a71-4aa3-8fda-bac0f5739a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:03.859921 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-2194dc0f-5a71-4aa3-8fda-bac0f5739a2c None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:03.861178 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-2194dc0f-5a71-4aa3-8fda-bac0f5739a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:03.891219 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:03.961968 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:04.248427 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8e2f4-833e-4eb6-8e1e-01737c1deb9b None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:04.250075 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-30a8e2f4-833e-4eb6-8e1e-01737c1deb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:04.355082 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8e2f4-833e-4eb6-8e1e-01737c1deb9b None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:04.384288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:04.431215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae17ed-ef98-407c-b05a-3782d92cca5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:04.431730 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae17ed-ef98-407c-b05a-3782d92cca5b service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:04.432023 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae17ed-ef98-407c-b05a-3782d92cca5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:04.432312 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae17ed-ef98-407c-b05a-3782d92cca5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:04.433933 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-fcae17ed-ef98-407c-b05a-3782d92cca5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:04.544635 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae17ed-ef98-407c-b05a-3782d92cca5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:04.546123 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae17ed-ef98-407c-b05a-3782d92cca5b service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:04.565145 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 735/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:06.553916 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c49eba-5f0f-4e32-8faa-38bee732d94f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:06.554479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c49eba-5f0f-4e32-8faa-38bee732d94f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:06.554479 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c49eba-5f0f-4e32-8faa-38bee732d94f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:06.555821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:06.561781 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c657f51-d02d-4e7b-9c85-e39b3252c255 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:06.562252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c657f51-d02d-4e7b-9c85-e39b3252c255 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:06.562357 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c657f51-d02d-4e7b-9c85-e39b3252c255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:06.629362 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-0c657f51-d02d-4e7b-9c85-e39b3252c255 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:06.630623 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0c657f51-d02d-4e7b-9c85-e39b3252c255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:06.639459 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:06.728812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:07.013853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c49eba-5f0f-4e32-8faa-38bee732d94f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:07.015436 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-90c49eba-5f0f-4e32-8faa-38bee732d94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:07.119109 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c49eba-5f0f-4e32-8faa-38bee732d94f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:07.174855 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:07.194579 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f714c-e45f-4306-9458-6b55561cb46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:07.195188 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f714c-e45f-4306-9458-6b55561cb46f service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:07.198779 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f714c-e45f-4306-9458-6b55561cb46f service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:07.198779 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f714c-e45f-4306-9458-6b55561cb46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:07.198779 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-616f714c-e45f-4306-9458-6b55561cb46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:07.308195 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f714c-e45f-4306-9458-6b55561cb46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:07.309582 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f714c-e45f-4306-9458-6b55561cb46f service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:07.327761 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 725/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:09.298283 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c06f8f-ff1c-4f1e-9909-9ab646d2ca3e None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:09.298838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c06f8f-ff1c-4f1e-9909-9ab646d2ca3e None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:09.298838 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c06f8f-ff1c-4f1e-9909-9ab646d2ca3e None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:09.300169 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:09.305819 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9488a-cee2-49d5-87aa-450b3e96815e None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:09.306417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9488a-cee2-49d5-87aa-450b3e96815e None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:09.306417 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9488a-cee2-49d5-87aa-450b3e96815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:09.377785 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-73e9488a-cee2-49d5-87aa-450b3e96815e None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:09.379219 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-73e9488a-cee2-49d5-87aa-450b3e96815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:09.419102 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:09.479698 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:09.762605 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c06f8f-ff1c-4f1e-9909-9ab646d2ca3e None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:09.764855 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-41c06f8f-ff1c-4f1e-9909-9ab646d2ca3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:09.869219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c06f8f-ff1c-4f1e-9909-9ab646d2ca3e None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:09.909266 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:09.945973 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbaec2-749b-42b6-88bc-6d3396aee0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:09.946584 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbaec2-749b-42b6-88bc-6d3396aee0fe service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:09.946866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbaec2-749b-42b6-88bc-6d3396aee0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:09.947141 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbaec2-749b-42b6-88bc-6d3396aee0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:09.948845 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-87bbaec2-749b-42b6-88bc-6d3396aee0fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:10.063435 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbaec2-749b-42b6-88bc-6d3396aee0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:10.064866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbaec2-749b-42b6-88bc-6d3396aee0fe service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:10.083602 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 738/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:12.053308 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2aac3-5454-4394-9fd4-a45f190002e1 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:12.053935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2aac3-5454-4394-9fd4-a45f190002e1 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:12.053935 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2aac3-5454-4394-9fd4-a45f190002e1 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:12.055259 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:12.060866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ba477-6d9f-4430-9b98-ab9166d4bf55 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:12.061395 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ba477-6d9f-4430-9b98-ab9166d4bf55 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:12.061395 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ba477-6d9f-4430-9b98-ab9166d4bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:12.127426 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-145ba477-6d9f-4430-9b98-ab9166d4bf55 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:12.128654 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-145ba477-6d9f-4430-9b98-ab9166d4bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:12.147102 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:12.226757 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:12.503035 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2aac3-5454-4394-9fd4-a45f190002e1 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:12.504598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-7ac2aac3-5454-4394-9fd4-a45f190002e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:12.612079 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2aac3-5454-4394-9fd4-a45f190002e1 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:12.677782 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:12.688719 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a637f7b-0b7c-490a-ac4e-2b832d552837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:12.697861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a637f7b-0b7c-490a-ac4e-2b832d552837 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:12.697861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a637f7b-0b7c-490a-ac4e-2b832d552837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:12.697861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a637f7b-0b7c-490a-ac4e-2b832d552837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:12.697861 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-6a637f7b-0b7c-490a-ac4e-2b832d552837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:12.800717 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a637f7b-0b7c-490a-ac4e-2b832d552837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:12.802200 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a637f7b-0b7c-490a-ac4e-2b832d552837 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:12.821048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 728/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:14.746425 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6fdf3-a2e8-4096-afb0-bde10c7e26da None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:14.746425 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6fdf3-a2e8-4096-afb0-bde10c7e26da None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:14.746425 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6fdf3-a2e8-4096-afb0-bde10c7e26da None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:14.746425 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 12:13:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:14.754904 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569e87f-68b8-416e-9c4b-62bd374ccd05 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:14.755308 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569e87f-68b8-416e-9c4b-62bd374ccd05 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:14.755471 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569e87f-68b8-416e-9c4b-62bd374ccd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:14.832596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-0569e87f-68b8-416e-9c4b-62bd374ccd05 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:14.833887 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-0569e87f-68b8-416e-9c4b-62bd374ccd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:14.885429 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:14.935373 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 12:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:15.214708 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6fdf3-a2e8-4096-afb0-bde10c7e26da None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:15.216290 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7af6fdf3-a2e8-4096-afb0-bde10c7e26da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:15.320852 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6fdf3-a2e8-4096-afb0-bde10c7e26da None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:15.369482 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:15.395729 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b863a57-2bcc-443c-b749-a020c881e6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:15.396336 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b863a57-2bcc-443c-b749-a020c881e6ef service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:15.396618 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b863a57-2bcc-443c-b749-a020c881e6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:15.396893 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b863a57-2bcc-443c-b749-a020c881e6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:15.398602 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-8b863a57-2bcc-443c-b749-a020c881e6ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:15.509013 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b863a57-2bcc-443c-b749-a020c881e6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:15.510710 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b863a57-2bcc-443c-b749-a020c881e6ef service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:15.530965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 741/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:24.933361 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=22 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:25.527970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=66 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:35.224330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d4164-f396-48c0-a490-725ded127d40 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:35.224904 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d4164-f396-48c0-a490-725ded127d40 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:35.224904 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d4164-f396-48c0-a490-725ded127d40 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:35.226311 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 730/1471] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:35.231583 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe620d-22b6-4b9c-ad87-b212d1e586f6 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:35.232636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe620d-22b6-4b9c-ad87-b212d1e586f6 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:35.232744 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe620d-22b6-4b9c-ad87-b212d1e586f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:35.298010 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-e0fe620d-22b6-4b9c-ad87-b212d1e586f6 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:35.299221 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-e0fe620d-22b6-4b9c-ad87-b212d1e586f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:35.395713 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 742/1472] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 12:13:35.694722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a12f8c-3c50-4017-bbe9-ab6d1c007fc5 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:35.697538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a12f8c-3c50-4017-bbe9-ab6d1c007fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:35.697538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a12f8c-3c50-4017-bbe9-ab6d1c007fc5 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:35.697538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 731/1473] 10.222.0.48 () {54 vars in 967 bytes} [Thu Sep 1 12:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:35.702081 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb169426-4242-43df-90bf-739a61526349 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:35.702377 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb169426-4242-43df-90bf-739a61526349 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:35.702657 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb169426-4242-43df-90bf-739a61526349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:35.779384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-bb169426-4242-43df-90bf-739a61526349 None None] MFA Rules not processed for user `bb79770a161740128e5792a70cbad00a`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:35.780613 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-bb169426-4242-43df-90bf-739a61526349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:35.800529 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:35.857551 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 743/1474] 10.222.0.48 () {58 vars in 1064 bytes} [Thu Sep 1 12:13:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:35.865147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a12f8c-3c50-4017-bbe9-ab6d1c007fc5 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:35.866697 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-e5a12f8c-3c50-4017-bbe9-ab6d1c007fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:35.973507 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a12f8c-3c50-4017-bbe9-ab6d1c007fc5 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:36.045419 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6017ae-14de-4954-9f2d-5b29b4009488 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb79770a161740128e5792a70cbad00a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:36.046058 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6017ae-14de-4954-9f2d-5b29b4009488 service glance] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:36.046335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6017ae-14de-4954-9f2d-5b29b4009488 service glance] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:36.046603 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6017ae-14de-4954-9f2d-5b29b4009488 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:36.048215 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-1b6017ae-14de-4954-9f2d-5b29b4009488 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:36.104094 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:36.159771 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6017ae-14de-4954-9f2d-5b29b4009488 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:36.161142 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6017ae-14de-4954-9f2d-5b29b4009488 service glance] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:36.180380 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 732/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:38.173361 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a4952-7701-40b1-a840-c39c6631bd0c None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:38.173970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a4952-7701-40b1-a840-c39c6631bd0c None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:38.173970 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a4952-7701-40b1-a840-c39c6631bd0c None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:38.175286 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 744/1476] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:38.181480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4428d-c061-43b2-b87c-9e54f44edfc2 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:38.182017 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4428d-c061-43b2-b87c-9e54f44edfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:38.182113 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4428d-c061-43b2-b87c-9e54f44edfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:38.249508 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-94c4428d-c061-43b2-b87c-9e54f44edfc2 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:38.250820 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-94c4428d-c061-43b2-b87c-9e54f44edfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:38.350481 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 733/1477] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:38.625365 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a4952-7701-40b1-a840-c39c6631bd0c None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:38.626998 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-a44a4952-7701-40b1-a840-c39c6631bd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:38.755409 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a4952-7701-40b1-a840-c39c6631bd0c None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:38.807391 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:38.852676 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62658f-c7c9-46b8-a38c-f6ad3dd0d9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:38.853284 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62658f-c7c9-46b8-a38c-f6ad3dd0d9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:38.853563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62658f-c7c9-46b8-a38c-f6ad3dd0d9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:38.853866 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62658f-c7c9-46b8-a38c-f6ad3dd0d9d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:38.854511 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62658f-c7c9-46b8-a38c-f6ad3dd0d9d0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:38.856313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WARNING py.warnings [None req-be62658f-c7c9-46b8-a38c-f6ad3dd0d9d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:13:38.856313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: warnings.warn(msg) Sep 01 12:13:38.856313 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: Sep 01 12:13:38.858112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62658f-c7c9-46b8-a38c-f6ad3dd0d9d0 admin admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:38.876894 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 745/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Sep 1 12:13:38 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 12:13:40.803570 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb53f4-8dc1-410a-b637-765f5408f3a8 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:40.804163 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb53f4-8dc1-410a-b637-765f5408f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:40.804163 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb53f4-8dc1-410a-b637-765f5408f3a8 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:40.805501 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 734/1479] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:40.810909 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c004b-fd7a-4a48-a57f-890860ab8a66 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:40.811415 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c004b-fd7a-4a48-a57f-890860ab8a66 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:40.811546 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c004b-fd7a-4a48-a57f-890860ab8a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:40.881470 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-694c004b-fd7a-4a48-a57f-890860ab8a66 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:40.882724 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-694c004b-fd7a-4a48-a57f-890860ab8a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:40.991490 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 746/1480] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:41.283250 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb53f4-8dc1-410a-b637-765f5408f3a8 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:41.284797 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9ecb53f4-8dc1-410a-b637-765f5408f3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:41.294796 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:41.434123 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb53f4-8dc1-410a-b637-765f5408f3a8 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:41.517618 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:41.551853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b86a4-5826-4099-b0fa-a6436280198a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8225879a66415c8a1dc28584e4c626', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:41.552405 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b86a4-5826-4099-b0fa-a6436280198a service nova] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:41.552940 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b86a4-5826-4099-b0fa-a6436280198a service nova] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:41.553216 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b86a4-5826-4099-b0fa-a6436280198a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:41.555024 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cc7b86a4-5826-4099-b0fa-a6436280198a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:41.648824 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b86a4-5826-4099-b0fa-a6436280198a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:41.650486 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b86a4-5826-4099-b0fa-a6436280198a service nova] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:41.668739 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 735/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:43.782398 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9110-c9a6-4949-ac68-23e25fd2fd2f None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:43.782977 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9110-c9a6-4949-ac68-23e25fd2fd2f None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:43.782977 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9110-c9a6-4949-ac68-23e25fd2fd2f None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:43.784354 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 747/1482] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:43.790026 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dda34d-55f6-41dc-b62d-dd37a15534e4 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:43.790692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dda34d-55f6-41dc-b62d-dd37a15534e4 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:43.790692 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dda34d-55f6-41dc-b62d-dd37a15534e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:43.858995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-05dda34d-55f6-41dc-b62d-dd37a15534e4 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:43.860226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-05dda34d-55f6-41dc-b62d-dd37a15534e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:43.906733 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:43.957914 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 736/1483] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:44.240349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd9110-c9a6-4949-ac68-23e25fd2fd2f None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:44.242028 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-6ffd9110-c9a6-4949-ac68-23e25fd2fd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:44.265424 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:44.347044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd9110-c9a6-4949-ac68-23e25fd2fd2f None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:44.420239 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538d3af-55ba-49da-aca3-eb7db9ebdd8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:44.420772 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538d3af-55ba-49da-aca3-eb7db9ebdd8d service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:44.421085 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538d3af-55ba-49da-aca3-eb7db9ebdd8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:44.421361 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538d3af-55ba-49da-aca3-eb7db9ebdd8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:44.423042 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-0538d3af-55ba-49da-aca3-eb7db9ebdd8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:44.498733 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:44.533743 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538d3af-55ba-49da-aca3-eb7db9ebdd8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:44.535349 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538d3af-55ba-49da-aca3-eb7db9ebdd8d service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:44.553728 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 748/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:46.520183 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372fbc4-3af9-4e26-a7ad-c05333c6a948 None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:46.520754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372fbc4-3af9-4e26-a7ad-c05333c6a948 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:46.520754 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372fbc4-3af9-4e26-a7ad-c05333c6a948 None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:46.522106 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 737/1485] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:46.527907 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a12a7-1f12-49fe-afb5-c5558fe62052 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:46.528413 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a12a7-1f12-49fe-afb5-c5558fe62052 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:46.534545 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a12a7-1f12-49fe-afb5-c5558fe62052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:46.605579 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-9b2a12a7-1f12-49fe-afb5-c5558fe62052 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:46.607469 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-9b2a12a7-1f12-49fe-afb5-c5558fe62052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:46.712252 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 749/1486] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:47.000421 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372fbc4-3af9-4e26-a7ad-c05333c6a948 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:47.002002 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-5372fbc4-3af9-4e26-a7ad-c05333c6a948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:47.227730 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372fbc4-3af9-4e26-a7ad-c05333c6a948 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:47.300368 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:47.302131 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84db25e-5257-4c7d-bf6a-adfc372b0d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:47.302995 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84db25e-5257-4c7d-bf6a-adfc372b0d2a service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:47.303274 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84db25e-5257-4c7d-bf6a-adfc372b0d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:47.303548 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84db25e-5257-4c7d-bf6a-adfc372b0d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:47.305167 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a84db25e-5257-4c7d-bf6a-adfc372b0d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:47.412842 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84db25e-5257-4c7d-bf6a-adfc372b0d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:47.414365 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84db25e-5257-4c7d-bf6a-adfc372b0d2a service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:47.433480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 738/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:49.448857 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f822eb-f586-4365-84e5-5bb8515dbfe8 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:49.449414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f822eb-f586-4365-84e5-5bb8515dbfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:49.449414 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f822eb-f586-4365-84e5-5bb8515dbfe8 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:49.451783 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 750/1488] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:49.457811 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8d454-5778-4074-945b-b2f931669dee None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:49.458360 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8d454-5778-4074-945b-b2f931669dee None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:49.458360 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8d454-5778-4074-945b-b2f931669dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:49.530087 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-c6c8d454-5778-4074-945b-b2f931669dee None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:49.531147 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-c6c8d454-5778-4074-945b-b2f931669dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:49.587632 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:49.632230 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 739/1489] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:49.923314 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f822eb-f586-4365-84e5-5bb8515dbfe8 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:49.924868 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-a9f822eb-f586-4365-84e5-5bb8515dbfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:49.963487 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:50.032833 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f822eb-f586-4365-84e5-5bb8515dbfe8 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:50.108288 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55edf5-e782-4c81-9b31-9c2b9223b195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:50.108896 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55edf5-e782-4c81-9b31-9c2b9223b195 service neutron] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:50.109180 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55edf5-e782-4c81-9b31-9c2b9223b195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:50.109453 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55edf5-e782-4c81-9b31-9c2b9223b195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:50.111107 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-cd55edf5-e782-4c81-9b31-9c2b9223b195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:50.191484 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:50.220401 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55edf5-e782-4c81-9b31-9c2b9223b195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:50.222342 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55edf5-e782-4c81-9b31-9c2b9223b195 service neutron] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:50.242060 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 751/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:13:52.593533 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1ce6b-35bc-4e48-b066-098806b2168f None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:52.594164 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1ce6b-35bc-4e48-b066-098806b2168f None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:52.594164 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1ce6b-35bc-4e48-b066-098806b2168f None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:52.595490 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 740/1491] 10.222.0.48 () {54 vars in 927 bytes} [Thu Sep 1 12:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:13:52.600698 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15797827-9d0c-4a1c-840a-bedb35336551 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:52.601168 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15797827-9d0c-4a1c-840a-bedb35336551 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:52.601276 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15797827-9d0c-4a1c-840a-bedb35336551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:52.670571 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-15797827-9d0c-4a1c-840a-bedb35336551 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:13:52.671865 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-15797827-9d0c-4a1c-840a-bedb35336551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:52.779178 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 752/1492] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Sep 1 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:13:53.066492 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1ce6b-35bc-4e48-b066-098806b2168f None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:13:53.068030 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-b9b1ce6b-35bc-4e48-b066-098806b2168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:53.172506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1ce6b-35bc-4e48-b066-098806b2168f None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:13:53.245071 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e8aca-dc99-4bcf-89a1-d9a442133ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:13:53.245702 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e8aca-dc99-4bcf-89a1-d9a442133ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:13:53.245977 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e8aca-dc99-4bcf-89a1-d9a442133ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:13:53.246244 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e8aca-dc99-4bcf-89a1-d9a442133ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:13:53.247839 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-a37e8aca-dc99-4bcf-89a1-d9a442133ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:13:53.257205 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:13:53.358553 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e8aca-dc99-4bcf-89a1-d9a442133ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:13:53.359922 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e8aca-dc99-4bcf-89a1-d9a442133ed5 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:13:53.380174 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 741/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:14:02.777215 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=82 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:14:03.377581 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=50 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:24.617245 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:24.617868 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:24.617868 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:24.700323 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:15:24.701628 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:24.803560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 753/1494] 10.222.0.48 () {58 vars in 958 bytes} [Thu Sep 1 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:15:24.812223 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209acbf6-1d30-418d-bd84-9b7406ee2f70 None None] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:24.812800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209acbf6-1d30-418d-bd84-9b7406ee2f70 None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:24.812800 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209acbf6-1d30-418d-bd84-9b7406ee2f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:24.891575 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.auth.core [None req-209acbf6-1d30-418d-bd84-9b7406ee2f70 None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81862) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:15:24.892804 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-209acbf6-1d30-418d-bd84-9b7406ee2f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:24.991229 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 742/1495] 10.222.0.48 () {58 vars in 958 bytes} [Thu Sep 1 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:15:24.996574 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:24.997869 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:25.009392 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757e171-e5d1-44df-8905-b6281e76c653 None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:25.084227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:25.119057 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:25.119964 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:25.119964 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:25.120279 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] PATH_INFO: `/v3/domains` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:25.121421 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81861) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 12:15:25.121754 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:25.122760 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WARNING py.warnings [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:25.122760 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: warnings.warn(msg) Sep 01 12:15:25.122760 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: Sep 01 12:15:25.123645 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:25.135135 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 754/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Sep 1 12:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 12:15:25.144241 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209acbf6-1d30-418d-bd84-9b7406ee2f70 None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:25.153794 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209acbf6-1d30-418d-bd84-9b7406ee2f70 None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:25.219006 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:25.254929 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:25.255454 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] REQUEST_METHOD: `POST` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:25.255722 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:25.255985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] PATH_INFO: `/v3/projects` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:25.256606 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:25.258382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:25.258382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Sep 01 12:15:25.258382 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: Sep 01 12:15:25.259571 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:25.384181 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 743/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Sep 1 12:15:25 2022] POST /identity/v3/projects => generated 379 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 12:15:25.388567 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:25.390163 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:25.425904 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1464ce-b25f-4dfd-a867-02a369503344 admin admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:25.523864 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:25.524399 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:25.525172 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:25.525465 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] PATH_INFO: `/v3/users` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:25.526153 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:25.527699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WARNING py.warnings [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:25.527699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: warnings.warn(msg) Sep 01 12:15:25.527699 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: Sep 01 12:15:25.528916 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:25.624563 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 755/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Sep 1 12:15:25 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 12:15:25.628752 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:25.630335 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:25.694268 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ea4e7-ed96-4706-97df-446d8a51237b admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:25.781138 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:25.791759 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:25.792398 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:25.792674 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:25.792943 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:25.793598 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:25.794678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:25.794678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Sep 01 12:15:25.794678 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: Sep 01 12:15:25.795596 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:25.809210 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 744/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Sep 1 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 12:15:25.813321 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:25.814922 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:25.827612 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4eb01-7192-4c9c-9a74-f2b74d549df0 admin admin] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:25.853620 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:25.925379 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeeb5e-fa2c-4ccd-b6cd-65fc0e0535cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:25.926027 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeeb5e-fa2c-4ccd-b6cd-65fc0e0535cb admin admin] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:25.926564 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeeb5e-fa2c-4ccd-b6cd-65fc0e0535cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:25.926836 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeeb5e-fa2c-4ccd-b6cd-65fc0e0535cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:25.927478 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeeb5e-fa2c-4ccd-b6cd-65fc0e0535cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:25.928492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WARNING py.warnings [None req-acbeeb5e-fa2c-4ccd-b6cd-65fc0e0535cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:25.928492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: warnings.warn(msg) Sep 01 12:15:25.928492 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: Sep 01 12:15:25.929384 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeeb5e-fa2c-4ccd-b6cd-65fc0e0535cb admin admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:25.942219 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 756/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Sep 1 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 12:15:25.946480 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:25.948039 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:25.983375 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7d5f-63e2-48c1-9f46-30b4aa907e4a admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:26.082666 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b4641-f375-4bc4-b0d2-84d9f35a8230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:26.083252 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b4641-f375-4bc4-b0d2-84d9f35a8230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:26.083523 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b4641-f375-4bc4-b0d2-84d9f35a8230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:26.083788 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b4641-f375-4bc4-b0d2-84d9f35a8230 admin admin] PATH_INFO: `/v3/projects/7e01adabca9149469ac658f6e6cbf9a3/users/e57a325d4a1246668d24c34606b964d6/roles/bddb19f21b7f472589b9154cb2aeeecc` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:26.106594 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b4641-f375-4bc4-b0d2-84d9f35a8230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e01adabca9149469ac658f6e6cbf9a3, user_id=e57a325d4a1246668d24c34606b964d6, role_id=bddb19f21b7f472589b9154cb2aeeecc)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:26.107630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-9c6b4641-f375-4bc4-b0d2-84d9f35a8230 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:26.107630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Sep 01 12:15:26.107630 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: Sep 01 12:15:26.109632 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b4641-f375-4bc4-b0d2-84d9f35a8230 admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:26.195606 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 745/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Sep 1 12:15:25 2022] PUT /identity/v3/projects/7e01adabca9149469ac658f6e6cbf9a3/users/e57a325d4a1246668d24c34606b964d6/roles/bddb19f21b7f472589b9154cb2aeeecc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 12:15:26.208937 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58088e2e-f3cf-4959-884e-7e682168f06d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:26.209227 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58088e2e-f3cf-4959-884e-7e682168f06d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:26.209503 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58088e2e-f3cf-4959-884e-7e682168f06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:26.268547 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-58088e2e-f3cf-4959-884e-7e682168f06d None None] MFA Rules not processed for user `e57a325d4a1246668d24c34606b964d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:15:26.269812 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-58088e2e-f3cf-4959-884e-7e682168f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:26.350110 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 757/1502] 10.222.0.48 () {58 vars in 958 bytes} [Thu Sep 1 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:15:26.363226 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ded1e-24f2-406a-90ea-3b48d2fadacf None None] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:26.363515 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ded1e-24f2-406a-90ea-3b48d2fadacf None None] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:26.363825 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ded1e-24f2-406a-90ea-3b48d2fadacf None None] PATH_INFO: `/` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:26.365175 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 746/1503] 10.222.0.48 () {54 vars in 964 bytes} [Thu Sep 1 12:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:15:26.370962 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b627-cdeb-4be7-a0b6-8fe21f4baeca None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:26.371278 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b627-cdeb-4be7-a0b6-8fe21f4baeca None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:26.371560 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b627-cdeb-4be7-a0b6-8fe21f4baeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:26.402952 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:26.455015 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-3487b627-cdeb-4be7-a0b6-8fe21f4baeca None None] MFA Rules not processed for user `86e8ec32ae1f46e9a1a3ada4fc7201c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:15:26.456287 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-3487b627-cdeb-4be7-a0b6-8fe21f4baeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:26.542858 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 758/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Sep 1 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:15:26.550741 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ded1e-24f2-406a-90ea-3b48d2fadacf None None] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:26.552538 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-689ded1e-24f2-406a-90ea-3b48d2fadacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:26.603587 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:26.658526 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ded1e-24f2-406a-90ea-3b48d2fadacf None None] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:26.738746 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86e8ec32ae1f46e9a1a3ada4fc7201c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:26.739370 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:26.739645 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:26.739917 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:26.741552 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:26.843228 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:26.844670 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:26.848140 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:26.863625 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 747/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:15:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 12:15:26.910531 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652bb4-22f8-4c14-b705-d6c6b5d0eab2 None None] REQUEST_METHOD: `GET` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:26.910965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652bb4-22f8-4c14-b705-d6c6b5d0eab2 None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:26.911089 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652bb4-22f8-4c14-b705-d6c6b5d0eab2 None None] PATH_INFO: `/` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:26.912350 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 759/1506] 10.222.0.48 () {52 vars in 830 bytes} [Thu Sep 1 12:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 12:15:26.915938 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:26.917189 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:27.017876 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a8fcb-f221-4160-bfa6-152eab2b8749 service cinder] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:27.111038 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:27.111663 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:27.111933 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:27.112196 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] PATH_INFO: `/v3/users/e57a325d4a1246668d24c34606b964d6` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:27.125882 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e57a325d4a1246668d24c34606b964d6)` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:27.126847 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: WARNING py.warnings [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:27.126847 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: warnings.warn(msg) Sep 01 12:15:27.126847 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: Sep 01 12:15:27.128160 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:27.138612 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=100 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:27.353658 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e57a325d4a1246668d24c34606b964d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:27.416506 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e57a325d4a1246668d24c34606b964d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:27.444611 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e57a325d4a1246668d24c34606b964d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:27.459701 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e57a325d4a1246668d24c34606b964d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:27.460297 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.notifications [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e57a325d4a1246668d24c34606b964d6'} {{(pid=81862) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:27.465985 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 748/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Sep 1 12:15:26 2022] DELETE /identity/v3/users/e57a325d4a1246668d24c34606b964d6 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 12:15:27.470591 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] REQUEST_METHOD: `POST` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:27.470965 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:27.471250 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:27.531647 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.auth.core [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] MFA Rules not processed for user `11b1bb7d9ce240aab79c21f829ee58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81861) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 12:15:27.532905 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:27.612653 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=100 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:27.625825 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 760/1508] 10.222.0.48 () {58 vars in 958 bytes} [Thu Sep 1 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 12:15:27.635634 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Authenticating user token {{(pid=81862) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:27.637012 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:27.747858 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8e58a-9102-4613-848f-85ef82eaedcb admin admin] Validating token access rules against request {{(pid=81862) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:27.806119 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:27.824853 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b1045-f2ba-4a7f-b3a0-9f37a268d056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e0ceaa2c6745dc89ade0c43a17572b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f9de6c1f1c645c095ec56f3ff7451da', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81862) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:27.825461 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b1045-f2ba-4a7f-b3a0-9f37a268d056 service neutron] REQUEST_METHOD: `GET` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:27.825821 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b1045-f2ba-4a7f-b3a0-9f37a268d056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:27.826093 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b1045-f2ba-4a7f-b3a0-9f37a268d056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81862) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:27.827690 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.fernet_utils [None req-8c4b1045-f2ba-4a7f-b3a0-9f37a268d056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81862) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:27.941048 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b1045-f2ba-4a7f-b3a0-9f37a268d056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:27.942560 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b1045-f2ba-4a7f-b3a0-9f37a268d056 service neutron] RBAC: Authorization granted {{(pid=81862) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:27.962396 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: [pid: 81862|app: 0|req: 749/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Sep 1 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 12:15:28.584688 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] Authenticating user token {{(pid=81861) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 12:15:28.585976 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.fernet_utils [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81861) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 12:15:28.597462 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe5e15-86b6-47c6-b01c-4e260a00af5d None None] Validating token access rules against request {{(pid=81861) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 12:15:28.716044 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11b1bb7d9ce240aab79c21f829ee58ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6c17f90daea4fc3b1b162a692b4ec70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81861) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 12:15:28.716704 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 12:15:28.717112 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 12:15:28.717262 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] PATH_INFO: `/v3/projects/7e01adabca9149469ac658f6e6cbf9a3` {{(pid=81861) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 12:15:28.729193 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e01adabca9149469ac658f6e6cbf9a3)` {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 12:15:28.730808 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: WARNING py.warnings [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 12:15:28.730808 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: warnings.warn(msg) Sep 01 12:15:28.730808 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: Sep 01 12:15:28.732096 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] RBAC: Authorization granted {{(pid=81861) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 12:15:28.863939 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e01adabca9149469ac658f6e6cbf9a3', 'request_id': 'req-643e7e08-56a6-4fa8-9821-e649bc9e1750'} {{(pid=81861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:28.870092 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e01adabca9149469ac658f6e6cbf9a3', 'request_id': 'req-643e7e08-56a6-4fa8-9821-e649bc9e1750'} {{(pid=81861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:28.965205 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e01adabca9149469ac658f6e6cbf9a3', 'request_id': 'req-643e7e08-56a6-4fa8-9821-e649bc9e1750'} {{(pid=81861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:28.998278 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] The token cache is being invalidate because project 7e01adabca9149469ac658f6e6cbf9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81861) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 12:15:28.998642 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:28.999257 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG keystone.notifications [None req-643e7e08-56a6-4fa8-9821-e649bc9e1750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81861) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 12:15:29.000636 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: [pid: 81861|app: 0|req: 761/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Sep 1 12:15:28 2022] DELETE /identity/v3/projects/7e01adabca9149469ac658f6e6cbf9a3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 12:15:38.012330 dv-cinder-839768-4-iscsi devstack@keystone.service[81862]: DEBUG dbcounter [-] [81862] Writing DB stats keystone:SELECT=62 {{(pid=81862) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 12:15:38.995824 dv-cinder-839768-4-iscsi devstack@keystone.service[81861]: DEBUG dbcounter [-] [81861] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81861) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}